35
Music Perception Fall 2002, Vol. 20, No. 1, 51–85 © 2002 BY THE REGENTS OF THE UNIVERSITY OF CALIFORNIA ALL RIGHTS RESERVED. 51 Address correspondence to Dirk-Jan Povel, NICI, P.O. Box 9104, 6500 HE Nijmegen. The Netherlands. (e-mail: [email protected]) ISSN: 0730-7829. Send requests for permission to reprint to Rights and Permissions, University of California Press, 2000 Center St., Ste. 303, Berkeley, CA 94704-1223. Harmonic Factors in the Perception of Tonal Melodies DIRK - JAN POVEL & ERIK JANSEN Nijmegen Institute for Information and Cognition By common assumption, the first step in processing a tonal melody con- sists in setting up the appropriate metrical and harmonic frames required for the mental representation of the sequence of tones. Focusing on the generation of a harmonic frame, this study aims (a) to discover the fac- tors that facilitate or interfere with the development of a harmonic inter- pretation, and (b) to test the hypothesis that goodness ratings of tone sequences largely depend on whether the listener succeeds in creating a suitable harmonic interpretation. In two experiments, listeners rated the melodic goodness of selected sequences of 10 and 13 tones and indicated which individual tones seemed not to fit. Results indicate that goodness ratings (a) are higher the more common the induced harmonic progres- sion, (b) are strongly affected by the occurrence and position of nonchord tones: sequences without nonchord tones were rated highest, sequences with anchoring nonchord tones intermediately, and nonanchoring nonchord tones lowest. The explanation offered is compared with pre- dictions derived from other theories, which leads to the conclusion that when a tone sequence is perceived as a melody, it is represented in terms of its underlying harmony, in which exact pitch-height characteristics play a minor role. Received September 20, 2001, accepted December 31, 2001. T HIS study is about the perception of tonal melodies and particularly the process by which the notes of simple tonal melodies are transformed into a musical percept. Numerous theoretical and empirical studies, some of which are discussed in detail here, have indicated the crucial importance of the development of an appropriate frame of reference, or schema, that identifies the context in which the tones in the input are interpreted and that guides further processing, possibly leading to a musical representation (e.g., Bharucha, 1987, 1991; Cuddy, Cohen, & Mewhort, 1981; Gjerdingen, 1990; Holleran, Jones, & Butler, 1995; Krumhansl, 1990; Krumhansl & Kessler, 1982; Lerdahl, 1988; Lerdahl & Jackendoff, 1983; Longuet-Higgins

2002Povel&Jansen

Embed Size (px)

DESCRIPTION

Nijmegen Institute for Information and Cognition Address correspondence to Dirk-Jan Povel, NICI, P.O. Box 9104, 6500 HE Nijmegen. The Netherlands. (e-mail: [email protected]) ISSN: 0730-7829. Send requests for permission to reprint to Rights and Permissions, University of California Press, 2000 Center St., Ste. 303, Berkeley, CA 94704-1223. 51 Music Perception Fall 2002, Vol. 20, No. 1, 51–85 © 2002 BY THE REGENTS OF THE UNIVERSITY OF CALIFORNIA ALL RIGHTS RESERVED. Dirk-Jan Povel & Erik Jansen 52

Citation preview

Page 1: 2002Povel&Jansen

Music PerceptionFall 2002 Vol 20 No 1 51ndash85

copy 2002 BY THE REGENTS OF THE UNIVERSITY OF CALIFORNIA

ALL RIGHTS RESERVED

51

Address correspondence to Dirk-Jan Povel NICI PO Box 9104 6500 HE NijmegenThe Netherlands (e-mail povelNICIkunnl)

ISSN 0730-7829 Send requests for permission to reprint to Rights and PermissionsUniversity of California Press 2000 Center St Ste 303 Berkeley CA 94704-1223

Harmonic Factors in the Perception of Tonal Melodies

D I R K - J A N P O V E L amp E R I K J A N S E N

Nijmegen Institute for Information and Cognition

By common assumption the first step in processing a tonal melody con-sists in setting up the appropriate metrical and harmonic frames requiredfor the mental representation of the sequence of tones Focusing on thegeneration of a harmonic frame this study aims (a) to discover the fac-tors that facilitate or interfere with the development of a harmonic inter-pretation and (b) to test the hypothesis that goodness ratings of tonesequences largely depend on whether the listener succeeds in creating asuitable harmonic interpretation In two experiments listeners rated themelodic goodness of selected sequences of 10 and 13 tones and indicatedwhich individual tones seemed not to fit Results indicate that goodnessratings (a) are higher the more common the induced harmonic progres-sion (b) are strongly affected by the occurrence and position of nonchordtones sequences without nonchord tones were rated highest sequenceswith anchoring nonchord tones intermediately and nonanchoringnonchord tones lowest The explanation offered is compared with pre-dictions derived from other theories which leads to the conclusion thatwhen a tone sequence is perceived as a melody it is represented in termsof its underlying harmony in which exact pitch-height characteristicsplay a minor role

Received September 20 2001 accepted December 31 2001

THIS study is about the perception of tonal melodies and particularly theprocess by which the notes of simple tonal melodies are transformed

into a musical percept Numerous theoretical and empirical studies someof which are discussed in detail here have indicated the crucial importanceof the development of an appropriate frame of reference or schema thatidentifies the context in which the tones in the input are interpreted andthat guides further processing possibly leading to a musical representation(eg Bharucha 1987 1991 Cuddy Cohen amp Mewhort 1981 Gjerdingen1990 Holleran Jones amp Butler 1995 Krumhansl 1990 Krumhansl ampKessler 1982 Lerdahl 1988 Lerdahl amp Jackendoff 1983 Longuet-Higgins

52 Dirk-Jan Povel amp Erik Jansen

amp Steedman 1971 Povel 1981 Povel amp Essens 1985 Sloboda amp Parker1985 Thompson 1993 Tillmann Bharucha amp Bigand 2000 Van DykeBingham 1910) For tonal music this frame of reference consists of a met-rical frame that enables the interpretation of the rhythmical aspects and aharmonic frame that allows the interpretation of the melodicharmonicaspects

The present study focuses on the induction of the harmonic frame and isbased on the assumption that a sequence of tones will be perceived as atonal melody only if the listener succeeds in discovering an acceptable un-derlying harmony By comparing the characteristics of sequences that areperceived as good or bad melodies we want to discover the factors thatdetermine whether the listener succeeds in creating an appropriate har-monic analysis and to develop hypotheses about how these sequences areprocessed Before dealing with the investigation proper we discuss a fewearlier studies that have greatly influenced the approach taken in the presentstudy

Van Dyke Bingham (1910) performed an experiment in which all me-lodic intervals within one octave were presented and subjects answeredthe question ldquoCan you make this second tone a final tone Does this melodyendrdquo (p 23) The results indicate that the descending perfect fifth thedescending major third and the ascending perfect fourth show the stron-gest tendency to be heard as final From a detailed analysis of all his resultshe concluded that ldquoTwo melodically lsquorelatedrsquo tones tend to establish atonality and the melody [the melodic interval] is judged to end only whenthe final tone is one of the members of the tonic triadmdashpreferably the tonicitselfrdquo (p 34) This is an interesting observation that indicates that evenwhen perceiving single melodic intervals listeners tend to interpret the tonesin a tonal frame by establishing a key

Sloboda and Parker (1985) reported an exploratory study in which eightsubjects four with musical training and four without provided six succes-sive sung recalls of a fragment of the Russian folksong ldquoSailorrdquo (compris-ing 30 notes) After transcription the reproductions were analyzed in sev-eral ways including a melodic contour analysis a metrical analysis arhythmical analysis a phrase structure analysis and a harmonic analysisThe main findings were that (a) recall of the melody was never perfecteven for the musically trained subjects (b) the metrical structure is alwayspreserved suggesting that meter is a primary structural frame for melodicrecall but the actual rhythms were in about half of the cases substituted bymetrical equivalents (c) the harmonic structure is coded but the exact me-lodic structure is often lost (d) musicians and nonmusicians appear to pro-cess the music in much the same way except for the harmonic relation-ships which are better coded by the musicians (e) subjectsrsquo performancedid not improve during the six trials on any of the measures

53Harmonic Factors in Melody Perception

The authors summarize the result of their study as follows ldquomemorizingsimple well-formed tonal melodies involves building a mental model ofthe underlying structure in which not all of the surface detail is necessarilyretainedrdquo (p 160) Thus even when subjects are explicitly asked to repro-duce a melody they appear to be unable to make a literal reproduction butrely on the generated underlying structure consisting of a metrical frameand a harmonic frame to generate a paraphrase The finding that the sub-jectsrsquo performance did not improve during the six trials is quite surprisingand may indicate that the exact coding of the melody is not part of theeveryday listening process

Research by Holleran Jones and Butler (1995) Platt and Racine (1994)and Thompson and Cuddy (1989) confirm the results of the study bySloboda and Parker (1985) by providing experimental evidence that listen-ers make a harmonic analysis if the input consists of a single voice melodyTrehub Thorpe and Trainor (1990) reported that infants 7ndash10 monthsold can detect small changes to tone patterns based on a V-I chord progres-sion better than changes to patterns not based on such a chord progressionsuggesting that even these very young children are sensitive to the underly-ing harmony (see however Trainor amp Trehub 1992)

Cuddy Cohen and Mewhort (1981) studied the perception of tone se-quences having ldquovarying degrees of musical structurerdquo They constructed aset of sequences by altering one or more tones of the ldquoprototypicalrdquo se-quence C5 E5 G5 F5 D5 B4 C5 (the numeral indicates the octave C4 = middleC) thereby gradually distorting the ldquoharmonic structurerdquo contour com-plexity and excursion size (interval between first and last tone) Based onthe results of Experiment 1 in which subjects judged the ldquotonality or tonestructurerdquo of 32 seven-tone sequences five levels of harmonic structurewere constructed by combining three rules (1) diatonicism (a series may ormay not consist of only diatonic tones) (2) leading-note-to-tonic ending(3) the extent to which a sequence follows a IndashVndashI harmonic progressionThese levels of harmonic structure were factorially combined with two lev-els of contour complexity and two levels of excursion yielding 20 stimulithat were recognized under transposition (Experiment 2) and rated on tonalstructure (Experiment 3) Findings indicate that the ratings were mostlyinfluenced by the factor harmonic structure and less by contour and excur-sion

Because the latter study has greatly influenced the research presentedhere we discuss a few of its aspects in more detail First the concept ofharmonic structure as expressed in the five levels of harmonic structure isnot theoretically but empirically determined As a result it is unclear howthe three rules have precisely determined the variable harmonic structureSecond although the rules describe listenersrsquo responses to the 20 sequencesof Experiments 2 and 3 reasonably well it is unclear to what extent the

54 Dirk-Jan Povel amp Erik Jansen

rules can be generalized to other tone sequences This is clarified with themelodic examples in Figure 1

Although Sequences 1a and 1b in Figure 1 both violate the rule ofdiatonicism Sequence 1b (containing two chromatic tones) will receive ahigher goodness rating than Sequence 1a most likely because the F and Drespectively resolve to the succeeding G and E Sequences 2b and 2c bothviolate the leading-tone-to-tonic-ending rule but they will probably be ratedequally high Finally Sequences 3b and 3c do not have an underlying IndashVndashI progression but they will still be judged good melodies or melodic frag-ments These examples do not undermine the general finding that harmonic

Fig 1 Examples of sequences that do not obey the harmonic rules of Cuddy Cohen andMewhort (1981) but still form good melodies 1a This sequence violates the diatonicismrule and indeed sounds bad 1b This sequence contains 2 violations of the diatonicism ruleyet it sounds good Sequences 2b and 2c both violate the leading tone-to-tonic rule yet theyare good melodies Sequences 3b and 3c both do not follow a IndashVndashI chord progression stillthey are good melodies

55Harmonic Factors in Melody Perception

structure is a major factor in the perception of tone sequences but theyindicate that their definition of harmonic structure is still incomplete

Next we want to advance some speculative explanations for why somesequences from the Cuddy et al (1981) study were judged as bad melodiesSome of the stimuli may have been rated low merely because they are expe-rienced as unfinished For example the sequence E5 B5 G5 F5 D5 A5 C5

(Stimulus 23 Expt 1 rating 29 on a 6-point scale1) can be transformedinto a good sequence by adding a few tones E5 B5 G 5 F5 D5 A5 C5 G4 B4 C5In a similar fashion the sequence C5 E5 B5 G5 F5 D5 A5 (Stimulus 18 Expt1 rating 32) can be transformed into a good one by extending it as fol-lows C5 E5 B5 G5 F5 D5 A5 G5 F5 B4 C5 The incompleteness of these twosequences thus seems to have been caused by specific implications createdby the succession of tones that are not met Note that both sequencescontain the fragment B5 G5 F5 D5 which is probably recognized as a G7chord implying a solution to elements of the tonic preferably the C whichdoes appear in the completed sequences Still another example is Stimulus11 (rating 44) C5 E5 G5 F5 D5 B4 A4 This sequence can very simply bealtered to form a good melody by adding either B4 C5 or G4 C5 The reasonthat this sequence received a higher rating than the previous two may bethat only two tones are needed to complete the sequence whereas in theformer ones more tones are needed for completion thereby putting morestrain on memory and imagination The same reasoning may be applied toStimulus 22 (rating 30) D5 F5 G 5 F 5 D 5 B4 C5 The fragment G 5 F 5 D 5

induces the G 7 chord creating an expectation for elements of the tonic Cbut the tones B and C do not fit these at all

The picture that arises from these descriptions is that the listener in hisher attempt to create a musical representation encounters a fragment thatinduces a musical interpretation (eg a V7 chord) and next determineswhether the following tones somehow fit with the expectation(s) createdby that interpretation If this happens to be the case the sequence will bejudged a good melody otherwise perception will fail and the sequence willbe judged a bad melody

Finally we consider Stimulus 13 (rating 43) C5 E5 G5 F 5 (G 5) D5 B4 C5This sequence can be repaired in two ways by invoking anchoring C

5 E

5 G

5

G 5 F5 D5 B4 C5 or C5 E5 G5 G 5 F5 D5 B4 G4 C5 The second sequence seems tosound better probably not because of harmonic but of rhythmical reasonsThe first sequence since it consists of eight tones cannot be given a metri-cal interpretation such that it ends on a downbeat The second sequencehowever consisting of nine tones can be conceived in a metrical frame byplacing downbeats on the first fifth and last tone Thus the last sequenceends on a downbeat and is metrically well-formed

1 The rating of the highly trained subjects is given here

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 2: 2002Povel&Jansen

52 Dirk-Jan Povel amp Erik Jansen

amp Steedman 1971 Povel 1981 Povel amp Essens 1985 Sloboda amp Parker1985 Thompson 1993 Tillmann Bharucha amp Bigand 2000 Van DykeBingham 1910) For tonal music this frame of reference consists of a met-rical frame that enables the interpretation of the rhythmical aspects and aharmonic frame that allows the interpretation of the melodicharmonicaspects

The present study focuses on the induction of the harmonic frame and isbased on the assumption that a sequence of tones will be perceived as atonal melody only if the listener succeeds in discovering an acceptable un-derlying harmony By comparing the characteristics of sequences that areperceived as good or bad melodies we want to discover the factors thatdetermine whether the listener succeeds in creating an appropriate har-monic analysis and to develop hypotheses about how these sequences areprocessed Before dealing with the investigation proper we discuss a fewearlier studies that have greatly influenced the approach taken in the presentstudy

Van Dyke Bingham (1910) performed an experiment in which all me-lodic intervals within one octave were presented and subjects answeredthe question ldquoCan you make this second tone a final tone Does this melodyendrdquo (p 23) The results indicate that the descending perfect fifth thedescending major third and the ascending perfect fourth show the stron-gest tendency to be heard as final From a detailed analysis of all his resultshe concluded that ldquoTwo melodically lsquorelatedrsquo tones tend to establish atonality and the melody [the melodic interval] is judged to end only whenthe final tone is one of the members of the tonic triadmdashpreferably the tonicitselfrdquo (p 34) This is an interesting observation that indicates that evenwhen perceiving single melodic intervals listeners tend to interpret the tonesin a tonal frame by establishing a key

Sloboda and Parker (1985) reported an exploratory study in which eightsubjects four with musical training and four without provided six succes-sive sung recalls of a fragment of the Russian folksong ldquoSailorrdquo (compris-ing 30 notes) After transcription the reproductions were analyzed in sev-eral ways including a melodic contour analysis a metrical analysis arhythmical analysis a phrase structure analysis and a harmonic analysisThe main findings were that (a) recall of the melody was never perfecteven for the musically trained subjects (b) the metrical structure is alwayspreserved suggesting that meter is a primary structural frame for melodicrecall but the actual rhythms were in about half of the cases substituted bymetrical equivalents (c) the harmonic structure is coded but the exact me-lodic structure is often lost (d) musicians and nonmusicians appear to pro-cess the music in much the same way except for the harmonic relation-ships which are better coded by the musicians (e) subjectsrsquo performancedid not improve during the six trials on any of the measures

53Harmonic Factors in Melody Perception

The authors summarize the result of their study as follows ldquomemorizingsimple well-formed tonal melodies involves building a mental model ofthe underlying structure in which not all of the surface detail is necessarilyretainedrdquo (p 160) Thus even when subjects are explicitly asked to repro-duce a melody they appear to be unable to make a literal reproduction butrely on the generated underlying structure consisting of a metrical frameand a harmonic frame to generate a paraphrase The finding that the sub-jectsrsquo performance did not improve during the six trials is quite surprisingand may indicate that the exact coding of the melody is not part of theeveryday listening process

Research by Holleran Jones and Butler (1995) Platt and Racine (1994)and Thompson and Cuddy (1989) confirm the results of the study bySloboda and Parker (1985) by providing experimental evidence that listen-ers make a harmonic analysis if the input consists of a single voice melodyTrehub Thorpe and Trainor (1990) reported that infants 7ndash10 monthsold can detect small changes to tone patterns based on a V-I chord progres-sion better than changes to patterns not based on such a chord progressionsuggesting that even these very young children are sensitive to the underly-ing harmony (see however Trainor amp Trehub 1992)

Cuddy Cohen and Mewhort (1981) studied the perception of tone se-quences having ldquovarying degrees of musical structurerdquo They constructed aset of sequences by altering one or more tones of the ldquoprototypicalrdquo se-quence C5 E5 G5 F5 D5 B4 C5 (the numeral indicates the octave C4 = middleC) thereby gradually distorting the ldquoharmonic structurerdquo contour com-plexity and excursion size (interval between first and last tone) Based onthe results of Experiment 1 in which subjects judged the ldquotonality or tonestructurerdquo of 32 seven-tone sequences five levels of harmonic structurewere constructed by combining three rules (1) diatonicism (a series may ormay not consist of only diatonic tones) (2) leading-note-to-tonic ending(3) the extent to which a sequence follows a IndashVndashI harmonic progressionThese levels of harmonic structure were factorially combined with two lev-els of contour complexity and two levels of excursion yielding 20 stimulithat were recognized under transposition (Experiment 2) and rated on tonalstructure (Experiment 3) Findings indicate that the ratings were mostlyinfluenced by the factor harmonic structure and less by contour and excur-sion

Because the latter study has greatly influenced the research presentedhere we discuss a few of its aspects in more detail First the concept ofharmonic structure as expressed in the five levels of harmonic structure isnot theoretically but empirically determined As a result it is unclear howthe three rules have precisely determined the variable harmonic structureSecond although the rules describe listenersrsquo responses to the 20 sequencesof Experiments 2 and 3 reasonably well it is unclear to what extent the

54 Dirk-Jan Povel amp Erik Jansen

rules can be generalized to other tone sequences This is clarified with themelodic examples in Figure 1

Although Sequences 1a and 1b in Figure 1 both violate the rule ofdiatonicism Sequence 1b (containing two chromatic tones) will receive ahigher goodness rating than Sequence 1a most likely because the F and Drespectively resolve to the succeeding G and E Sequences 2b and 2c bothviolate the leading-tone-to-tonic-ending rule but they will probably be ratedequally high Finally Sequences 3b and 3c do not have an underlying IndashVndashI progression but they will still be judged good melodies or melodic frag-ments These examples do not undermine the general finding that harmonic

Fig 1 Examples of sequences that do not obey the harmonic rules of Cuddy Cohen andMewhort (1981) but still form good melodies 1a This sequence violates the diatonicismrule and indeed sounds bad 1b This sequence contains 2 violations of the diatonicism ruleyet it sounds good Sequences 2b and 2c both violate the leading tone-to-tonic rule yet theyare good melodies Sequences 3b and 3c both do not follow a IndashVndashI chord progression stillthey are good melodies

55Harmonic Factors in Melody Perception

structure is a major factor in the perception of tone sequences but theyindicate that their definition of harmonic structure is still incomplete

Next we want to advance some speculative explanations for why somesequences from the Cuddy et al (1981) study were judged as bad melodiesSome of the stimuli may have been rated low merely because they are expe-rienced as unfinished For example the sequence E5 B5 G5 F5 D5 A5 C5

(Stimulus 23 Expt 1 rating 29 on a 6-point scale1) can be transformedinto a good sequence by adding a few tones E5 B5 G 5 F5 D5 A5 C5 G4 B4 C5In a similar fashion the sequence C5 E5 B5 G5 F5 D5 A5 (Stimulus 18 Expt1 rating 32) can be transformed into a good one by extending it as fol-lows C5 E5 B5 G5 F5 D5 A5 G5 F5 B4 C5 The incompleteness of these twosequences thus seems to have been caused by specific implications createdby the succession of tones that are not met Note that both sequencescontain the fragment B5 G5 F5 D5 which is probably recognized as a G7chord implying a solution to elements of the tonic preferably the C whichdoes appear in the completed sequences Still another example is Stimulus11 (rating 44) C5 E5 G5 F5 D5 B4 A4 This sequence can very simply bealtered to form a good melody by adding either B4 C5 or G4 C5 The reasonthat this sequence received a higher rating than the previous two may bethat only two tones are needed to complete the sequence whereas in theformer ones more tones are needed for completion thereby putting morestrain on memory and imagination The same reasoning may be applied toStimulus 22 (rating 30) D5 F5 G 5 F 5 D 5 B4 C5 The fragment G 5 F 5 D 5

induces the G 7 chord creating an expectation for elements of the tonic Cbut the tones B and C do not fit these at all

The picture that arises from these descriptions is that the listener in hisher attempt to create a musical representation encounters a fragment thatinduces a musical interpretation (eg a V7 chord) and next determineswhether the following tones somehow fit with the expectation(s) createdby that interpretation If this happens to be the case the sequence will bejudged a good melody otherwise perception will fail and the sequence willbe judged a bad melody

Finally we consider Stimulus 13 (rating 43) C5 E5 G5 F 5 (G 5) D5 B4 C5This sequence can be repaired in two ways by invoking anchoring C

5 E

5 G

5

G 5 F5 D5 B4 C5 or C5 E5 G5 G 5 F5 D5 B4 G4 C5 The second sequence seems tosound better probably not because of harmonic but of rhythmical reasonsThe first sequence since it consists of eight tones cannot be given a metri-cal interpretation such that it ends on a downbeat The second sequencehowever consisting of nine tones can be conceived in a metrical frame byplacing downbeats on the first fifth and last tone Thus the last sequenceends on a downbeat and is metrically well-formed

1 The rating of the highly trained subjects is given here

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 3: 2002Povel&Jansen

53Harmonic Factors in Melody Perception

The authors summarize the result of their study as follows ldquomemorizingsimple well-formed tonal melodies involves building a mental model ofthe underlying structure in which not all of the surface detail is necessarilyretainedrdquo (p 160) Thus even when subjects are explicitly asked to repro-duce a melody they appear to be unable to make a literal reproduction butrely on the generated underlying structure consisting of a metrical frameand a harmonic frame to generate a paraphrase The finding that the sub-jectsrsquo performance did not improve during the six trials is quite surprisingand may indicate that the exact coding of the melody is not part of theeveryday listening process

Research by Holleran Jones and Butler (1995) Platt and Racine (1994)and Thompson and Cuddy (1989) confirm the results of the study bySloboda and Parker (1985) by providing experimental evidence that listen-ers make a harmonic analysis if the input consists of a single voice melodyTrehub Thorpe and Trainor (1990) reported that infants 7ndash10 monthsold can detect small changes to tone patterns based on a V-I chord progres-sion better than changes to patterns not based on such a chord progressionsuggesting that even these very young children are sensitive to the underly-ing harmony (see however Trainor amp Trehub 1992)

Cuddy Cohen and Mewhort (1981) studied the perception of tone se-quences having ldquovarying degrees of musical structurerdquo They constructed aset of sequences by altering one or more tones of the ldquoprototypicalrdquo se-quence C5 E5 G5 F5 D5 B4 C5 (the numeral indicates the octave C4 = middleC) thereby gradually distorting the ldquoharmonic structurerdquo contour com-plexity and excursion size (interval between first and last tone) Based onthe results of Experiment 1 in which subjects judged the ldquotonality or tonestructurerdquo of 32 seven-tone sequences five levels of harmonic structurewere constructed by combining three rules (1) diatonicism (a series may ormay not consist of only diatonic tones) (2) leading-note-to-tonic ending(3) the extent to which a sequence follows a IndashVndashI harmonic progressionThese levels of harmonic structure were factorially combined with two lev-els of contour complexity and two levels of excursion yielding 20 stimulithat were recognized under transposition (Experiment 2) and rated on tonalstructure (Experiment 3) Findings indicate that the ratings were mostlyinfluenced by the factor harmonic structure and less by contour and excur-sion

Because the latter study has greatly influenced the research presentedhere we discuss a few of its aspects in more detail First the concept ofharmonic structure as expressed in the five levels of harmonic structure isnot theoretically but empirically determined As a result it is unclear howthe three rules have precisely determined the variable harmonic structureSecond although the rules describe listenersrsquo responses to the 20 sequencesof Experiments 2 and 3 reasonably well it is unclear to what extent the

54 Dirk-Jan Povel amp Erik Jansen

rules can be generalized to other tone sequences This is clarified with themelodic examples in Figure 1

Although Sequences 1a and 1b in Figure 1 both violate the rule ofdiatonicism Sequence 1b (containing two chromatic tones) will receive ahigher goodness rating than Sequence 1a most likely because the F and Drespectively resolve to the succeeding G and E Sequences 2b and 2c bothviolate the leading-tone-to-tonic-ending rule but they will probably be ratedequally high Finally Sequences 3b and 3c do not have an underlying IndashVndashI progression but they will still be judged good melodies or melodic frag-ments These examples do not undermine the general finding that harmonic

Fig 1 Examples of sequences that do not obey the harmonic rules of Cuddy Cohen andMewhort (1981) but still form good melodies 1a This sequence violates the diatonicismrule and indeed sounds bad 1b This sequence contains 2 violations of the diatonicism ruleyet it sounds good Sequences 2b and 2c both violate the leading tone-to-tonic rule yet theyare good melodies Sequences 3b and 3c both do not follow a IndashVndashI chord progression stillthey are good melodies

55Harmonic Factors in Melody Perception

structure is a major factor in the perception of tone sequences but theyindicate that their definition of harmonic structure is still incomplete

Next we want to advance some speculative explanations for why somesequences from the Cuddy et al (1981) study were judged as bad melodiesSome of the stimuli may have been rated low merely because they are expe-rienced as unfinished For example the sequence E5 B5 G5 F5 D5 A5 C5

(Stimulus 23 Expt 1 rating 29 on a 6-point scale1) can be transformedinto a good sequence by adding a few tones E5 B5 G 5 F5 D5 A5 C5 G4 B4 C5In a similar fashion the sequence C5 E5 B5 G5 F5 D5 A5 (Stimulus 18 Expt1 rating 32) can be transformed into a good one by extending it as fol-lows C5 E5 B5 G5 F5 D5 A5 G5 F5 B4 C5 The incompleteness of these twosequences thus seems to have been caused by specific implications createdby the succession of tones that are not met Note that both sequencescontain the fragment B5 G5 F5 D5 which is probably recognized as a G7chord implying a solution to elements of the tonic preferably the C whichdoes appear in the completed sequences Still another example is Stimulus11 (rating 44) C5 E5 G5 F5 D5 B4 A4 This sequence can very simply bealtered to form a good melody by adding either B4 C5 or G4 C5 The reasonthat this sequence received a higher rating than the previous two may bethat only two tones are needed to complete the sequence whereas in theformer ones more tones are needed for completion thereby putting morestrain on memory and imagination The same reasoning may be applied toStimulus 22 (rating 30) D5 F5 G 5 F 5 D 5 B4 C5 The fragment G 5 F 5 D 5

induces the G 7 chord creating an expectation for elements of the tonic Cbut the tones B and C do not fit these at all

The picture that arises from these descriptions is that the listener in hisher attempt to create a musical representation encounters a fragment thatinduces a musical interpretation (eg a V7 chord) and next determineswhether the following tones somehow fit with the expectation(s) createdby that interpretation If this happens to be the case the sequence will bejudged a good melody otherwise perception will fail and the sequence willbe judged a bad melody

Finally we consider Stimulus 13 (rating 43) C5 E5 G5 F 5 (G 5) D5 B4 C5This sequence can be repaired in two ways by invoking anchoring C

5 E

5 G

5

G 5 F5 D5 B4 C5 or C5 E5 G5 G 5 F5 D5 B4 G4 C5 The second sequence seems tosound better probably not because of harmonic but of rhythmical reasonsThe first sequence since it consists of eight tones cannot be given a metri-cal interpretation such that it ends on a downbeat The second sequencehowever consisting of nine tones can be conceived in a metrical frame byplacing downbeats on the first fifth and last tone Thus the last sequenceends on a downbeat and is metrically well-formed

1 The rating of the highly trained subjects is given here

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 4: 2002Povel&Jansen

54 Dirk-Jan Povel amp Erik Jansen

rules can be generalized to other tone sequences This is clarified with themelodic examples in Figure 1

Although Sequences 1a and 1b in Figure 1 both violate the rule ofdiatonicism Sequence 1b (containing two chromatic tones) will receive ahigher goodness rating than Sequence 1a most likely because the F and Drespectively resolve to the succeeding G and E Sequences 2b and 2c bothviolate the leading-tone-to-tonic-ending rule but they will probably be ratedequally high Finally Sequences 3b and 3c do not have an underlying IndashVndashI progression but they will still be judged good melodies or melodic frag-ments These examples do not undermine the general finding that harmonic

Fig 1 Examples of sequences that do not obey the harmonic rules of Cuddy Cohen andMewhort (1981) but still form good melodies 1a This sequence violates the diatonicismrule and indeed sounds bad 1b This sequence contains 2 violations of the diatonicism ruleyet it sounds good Sequences 2b and 2c both violate the leading tone-to-tonic rule yet theyare good melodies Sequences 3b and 3c both do not follow a IndashVndashI chord progression stillthey are good melodies

55Harmonic Factors in Melody Perception

structure is a major factor in the perception of tone sequences but theyindicate that their definition of harmonic structure is still incomplete

Next we want to advance some speculative explanations for why somesequences from the Cuddy et al (1981) study were judged as bad melodiesSome of the stimuli may have been rated low merely because they are expe-rienced as unfinished For example the sequence E5 B5 G5 F5 D5 A5 C5

(Stimulus 23 Expt 1 rating 29 on a 6-point scale1) can be transformedinto a good sequence by adding a few tones E5 B5 G 5 F5 D5 A5 C5 G4 B4 C5In a similar fashion the sequence C5 E5 B5 G5 F5 D5 A5 (Stimulus 18 Expt1 rating 32) can be transformed into a good one by extending it as fol-lows C5 E5 B5 G5 F5 D5 A5 G5 F5 B4 C5 The incompleteness of these twosequences thus seems to have been caused by specific implications createdby the succession of tones that are not met Note that both sequencescontain the fragment B5 G5 F5 D5 which is probably recognized as a G7chord implying a solution to elements of the tonic preferably the C whichdoes appear in the completed sequences Still another example is Stimulus11 (rating 44) C5 E5 G5 F5 D5 B4 A4 This sequence can very simply bealtered to form a good melody by adding either B4 C5 or G4 C5 The reasonthat this sequence received a higher rating than the previous two may bethat only two tones are needed to complete the sequence whereas in theformer ones more tones are needed for completion thereby putting morestrain on memory and imagination The same reasoning may be applied toStimulus 22 (rating 30) D5 F5 G 5 F 5 D 5 B4 C5 The fragment G 5 F 5 D 5

induces the G 7 chord creating an expectation for elements of the tonic Cbut the tones B and C do not fit these at all

The picture that arises from these descriptions is that the listener in hisher attempt to create a musical representation encounters a fragment thatinduces a musical interpretation (eg a V7 chord) and next determineswhether the following tones somehow fit with the expectation(s) createdby that interpretation If this happens to be the case the sequence will bejudged a good melody otherwise perception will fail and the sequence willbe judged a bad melody

Finally we consider Stimulus 13 (rating 43) C5 E5 G5 F 5 (G 5) D5 B4 C5This sequence can be repaired in two ways by invoking anchoring C

5 E

5 G

5

G 5 F5 D5 B4 C5 or C5 E5 G5 G 5 F5 D5 B4 G4 C5 The second sequence seems tosound better probably not because of harmonic but of rhythmical reasonsThe first sequence since it consists of eight tones cannot be given a metri-cal interpretation such that it ends on a downbeat The second sequencehowever consisting of nine tones can be conceived in a metrical frame byplacing downbeats on the first fifth and last tone Thus the last sequenceends on a downbeat and is metrically well-formed

1 The rating of the highly trained subjects is given here

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 5: 2002Povel&Jansen

55Harmonic Factors in Melody Perception

structure is a major factor in the perception of tone sequences but theyindicate that their definition of harmonic structure is still incomplete

Next we want to advance some speculative explanations for why somesequences from the Cuddy et al (1981) study were judged as bad melodiesSome of the stimuli may have been rated low merely because they are expe-rienced as unfinished For example the sequence E5 B5 G5 F5 D5 A5 C5

(Stimulus 23 Expt 1 rating 29 on a 6-point scale1) can be transformedinto a good sequence by adding a few tones E5 B5 G 5 F5 D5 A5 C5 G4 B4 C5In a similar fashion the sequence C5 E5 B5 G5 F5 D5 A5 (Stimulus 18 Expt1 rating 32) can be transformed into a good one by extending it as fol-lows C5 E5 B5 G5 F5 D5 A5 G5 F5 B4 C5 The incompleteness of these twosequences thus seems to have been caused by specific implications createdby the succession of tones that are not met Note that both sequencescontain the fragment B5 G5 F5 D5 which is probably recognized as a G7chord implying a solution to elements of the tonic preferably the C whichdoes appear in the completed sequences Still another example is Stimulus11 (rating 44) C5 E5 G5 F5 D5 B4 A4 This sequence can very simply bealtered to form a good melody by adding either B4 C5 or G4 C5 The reasonthat this sequence received a higher rating than the previous two may bethat only two tones are needed to complete the sequence whereas in theformer ones more tones are needed for completion thereby putting morestrain on memory and imagination The same reasoning may be applied toStimulus 22 (rating 30) D5 F5 G 5 F 5 D 5 B4 C5 The fragment G 5 F 5 D 5

induces the G 7 chord creating an expectation for elements of the tonic Cbut the tones B and C do not fit these at all

The picture that arises from these descriptions is that the listener in hisher attempt to create a musical representation encounters a fragment thatinduces a musical interpretation (eg a V7 chord) and next determineswhether the following tones somehow fit with the expectation(s) createdby that interpretation If this happens to be the case the sequence will bejudged a good melody otherwise perception will fail and the sequence willbe judged a bad melody

Finally we consider Stimulus 13 (rating 43) C5 E5 G5 F 5 (G 5) D5 B4 C5This sequence can be repaired in two ways by invoking anchoring C

5 E

5 G

5

G 5 F5 D5 B4 C5 or C5 E5 G5 G 5 F5 D5 B4 G4 C5 The second sequence seems tosound better probably not because of harmonic but of rhythmical reasonsThe first sequence since it consists of eight tones cannot be given a metri-cal interpretation such that it ends on a downbeat The second sequencehowever consisting of nine tones can be conceived in a metrical frame byplacing downbeats on the first fifth and last tone Thus the last sequenceends on a downbeat and is metrically well-formed

1 The rating of the highly trained subjects is given here

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 6: 2002Povel&Jansen

56 Dirk-Jan Povel amp Erik Jansen

From these examples it may be concluded that the goodness of a se-quence is determined both by harmonic and metrical factors the harmonicinterpretation must be such that implications are resolved and the metricalinterpretation should allow the organization of the rhythmical content Theseideas will be elaborated on further later

Povel and Jansen (2001) and Jansen and Povel (2000) performed a seriesof studies in which listeners judged the goodness of different sets of tonesequences (a) tone sequences containing both diatonic and chromatic tones(b) tone sequences containing only diatonic tones and (c) tone sequencescontaining only arpeggiated chords These studies are reviewed in Poveland Jansen (2000) Povel and Jansen (2001) studied the perception of a setof tone sequences consisting of a subset of all orderings of the collection C4

E4 F 4 G4 B 4 Each sequence was preceded by the chords C7ndashF to induce thekey of F major It was hypothesized that a tone series is judged a good melodyif either one or both of the perceptual mechanisms chord recognition andanchoring (Bharucha 1984 1996) can be applied to the series Chord recog-nition is the mechanism that describes a series of tones as a chord and anchor-ing is the mechanism that links a tone to a (chord) tone occurring later in theseries Applying these mechanisms a sequence of tones is conceived as thechord C7 if the F 4 which does not belong to the chord can be ldquoanchoredrdquo toa subsequent G4 Anchoring may either be immediate when the G follows theF as in the tone series C

4 E

4 F

4 G

4 B

4 or more or less delayed when one or

more tones intervene between the F and G as in the series E4 F 4 C4 G4 B 4 orB 4 F 4 E4 C4 G4 Experiments in which listeners rated melodic goodness andproduced the expectations created at different positions in the sequence sup-ported the hypothesis The same hypothesis was tested for sequences contain-ing only diatonic tones notably a subset of 60 sequences from the set contain-ing all orderings of the collection D

4 E

4 F

4 G

4 A

4 B

4 (Jansen amp Povel 1999)

Although the responses showed considerable interindividual differences theresults globally supported the hypothesis

Jansen and Povel (2000) studied sequences consisting of arpeggiatedchords Thirty-two six-tone sequences were constructed each metricallysegmented into two groups of three Each group consisted of tones fromone of the triads I IV or V Chord progressions were formed by four differ-ent combinations of these triads Contour complexity of the sequences wasalso manipulated Ratings of the tone sequences indicate that the goodnessresponses are determined by the usualness of the perceived implied har-monic progression (eg IndashIV being rated higher than VndashIV) as well as bythe contour complexity of the sequences

THE PROCESSING OF SIMPLE TONAL MELODIES

If we combine the results of the studies just discussed the followingconceptualization of the processing of tonal melodies emerges Upon hear-

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 7: 2002Povel&Jansen

57Harmonic Factors in Melody Perception

ing the initial tones of a sequence the listener attempts to establish theinterpretational context of the sequence the metrical frame and the har-monic frame As this study focuses on the induction of the harmonic framewe do not discuss the metrical aspect of the interpretational context here(all tone sequences used in the experiments induce the same meter) Theharmonic frame has a global and a local aspect the global context consistsof a key and a mode whereas the local context consists of a region withinthe key (I V vi etc) After the listener has established key and mode (s)hewill attempt to divide the sequence into regions each associated with aharmony How the processes of key finding and region assignment pre-cisely interact is not well understood Although the system is usually con-ceived as completely hierarchical (eg Bharucha 1987 Tillmann Bharuchaamp Bigand 2000) some authors have argued for a partially hierarchicalsystem (Povel amp Van Egmond 1993 Thompson 1993 Thompson ampCuddy 1989) The harmonic function of the region determines the musicalfunction of the tones within that region thereby determining the stabilityof the tones and setting up expectations for resolutions of unstable tonesThe regions themselves also differ in stability and also create expectationsfor succeeding regions Thus the harmonic analysis has two levels on thehighest level functions the key on the lower level the region

Whether the listener succeeds in making a harmonic analysis largely de-pends on the distribution of the tones and especially on the occurrence andlocalization of nonchord tones (nonchord tones are of course only definedif surrounding tones are interpreted as chord tones) Nonchord tones in-deed form a central problem in the development of an algorithmic har-monic analysis (Temperley 1997) Nonchord tones can be incorporated ina harmonic analysis (a) if they can be linked to a following chord tone(anchoring) or (b) if they are assimilated in a run of steps (a series of minoror major seconds) If the harmonic analysis has been successful (meaningthat nonchord tones are somehow accommodated) the listener determineswhether the chordal implications of the activated regions are resolved (whichwill generally be the case if the harmonic progression follows Pistonrsquos table[Piston amp Devoto 1989] of usual root progressions) It is assumed thatonly as long as the metrical and harmonic interpretation is successful iscomplete processing of the sequence (eg the coding of the actual rhythmi-cal figures and pitch patterns) possible

The ideas proposed here are illustrated with a few melodic examples(Figure 2) The supposed goodness (or badness) of these sequences will beexplained in terms of these ideas The fact that the A in Sequence a does notseem to fit can be explained in two ways (a) the A does not fit in the G7chord which is induced by the G F and D (which would be resolved by thesubsequent C) (b) upon hearing the A the tones F D A induce the region iiresulting in the progression IndashiindashI which is rather unusual These two hy-potheses are put to the test in the next melodic examples In Sequence b

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 8: 2002Povel&Jansen

58 Dirk-Jan Povel amp Erik Jansen

the A is incorporated in the G7 chord by means of the added tone B towhich the A can be anchored In Sequence c the ii chord is followed by aV7 chord yielding the common progression IndashiindashV7ndashI Sequence d finallyshows how the A is incorporated in a scale fragment or run of steps

THE PRESENT RESEARCH

On the basis of the studies just described and the emanating theoreticalconsiderations proposed we can now list the major reasons why a listenerjudges a tone sequence as not being a good melody These are (a) the se-quence is metrically indeterminate the listener is unable to make a suitablemetrical interpretation for instance because the sequence does not end ona downbeat etc (b) the sequence is harmonically indeterminate the lis-tener is unable to make a coherent harmonic interpretation (c) the sequenceinduces an implied harmony with an irregular harmonic rhythm (d) thesequence is not finished that is the perceptual processing of the sequencehas created one or more expectations (implications) that remain unresolved(e) the contour of the sequence is relatively complex (Cuddy et al 1981Jansen amp Povel 1999) and (f) the first and last tone of the sequence arenot the same (Cuddy et al 1981)

Earlier research on rhythm perception showed that the discovery of anunderlying meter is a prerequisite for the formation of an accurate mentalrepresentation of a rhythmical pattern (Essens amp Povel 1985 Povel 1981Povel amp Essens 1985) Analogously in the present research we focus onthe harmonic interpretation and test the hypothesis that a tone sequencecan lead to a musical percept only if the listener succeeds in discovering the

Fig 2 Harmonic factors that determine melodic goodness (see text for explanation)

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 9: 2002Povel&Jansen

59Harmonic Factors in Melody Perception

underlying harmonic frame The main question we hope to answer is whatthe conditions are for a successful harmonic analysis The exact hypothesisreads A tone sequence will lead to a musical percept only if (a) the at-tempt to make a harmonic analysis succeeds and (b) the induced harmonicprogression consists of a succession of regions in which the harmonic ex-pectations are resolved The predictions of these hypotheses are schemati-cally represented in Figure 3

COMPARISON WITH ALTERNATIVE EXPLANATIONS

The explanation just proposed which capitalizes on the creation of aharmonic frame in the process of developing a musical representation willbe compared with three alternative explanations notably (a) predictionsderived from the tonal hierarchy model of Krumhansl (1990 Krumhanslamp Kessler 1982) (b) predictions derived from the theory of Narmour(1990) and (c) predictions based on specific features in the sequence

Predictions Derived from the Tonal Hierarchy Model

According to the tonal hierarchy model the tones in a key differ in theirldquostabilityrdquo the tonic being most stable followed by the other tones of thetonic triad the diatonic tones with the nondiatonic tones being the leaststable This variable called ldquotonalityrdquo has been investigated in studies byCuddy and Lunney (1995) Krumhansl (1995) and Schellenberg (19961997) and shown to play a role in listenersrsquo ratings of how well a third tonecontinues a two-tone sequence On the assumption that a sequence willform a better melody as its tones have a higher average stability we have

Fig 3 Schematic representation of the predictions made in Experiment 1

Harmonic analysissuccesful

no

no

yes

yes

No tonal melody

Bad melody Good melody

Common progression

Harmonic analysissuccessful

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 10: 2002Povel&Jansen

60 Dirk-Jan Povel amp Erik Jansen

calculated a variable MeanStab which is the mean of the stabilities of thetones in the sequence

Predictions Derived from Narmourrsquos Theory

Narmourrsquos implication-realization model (Narmour 1990 1992) assumesthat the expectations listeners form when they hear a melody are based ona limited number of factors that are partly innate and partly learned Herewe focus on the innate factors which are related to the Gestalt principlesof proximity similarity and symmetry Central in the implication-realiza-tion model are the concepts of an implicative interval (consisting of twotones) followed by a realized interval (consisting of the last tone of theimplicative interval and the following tone) According to the theory animplicative interval implies that some tones are more likely to follow thanothers which signifies that some successions of implicative and realizedintervals (strings of three tones) are more expected than others One com-ponent of the implication-realization model namely that which deals withthe innate expectations created by contour characteristics was formalizedby Schellenberg (1996) in terms of five variables registral direction inter-vallic difference registral return proximity and closure

The variables are described here only briefly for a more detailed de-scription see Krumhansl (1995) or Schellenberg (1996) The first two prin-ciples which form the core of the theory depend on the size of the implica-tive interval small (5 semitones or less) or large (7 semitones or more) Thetritone (6 semitones) is considered a threshold interval being neither smallnor large

1 The principle of registral direction states that a small implicativeinterval implies a continuation of pitch direction whereas a largeinterval implies a change of direction

2 The principle of intervallic difference states that a small implica-tive interval implies a similarly sized realized interval whereas alarge implicative interval implies a relatively smaller realized in-terval

3 The principle of registral return describes an archetypical me-lodic form namely the tendency for the second tone of the real-ized interval to be proximate in pitch to the first tone of theimplicative interval (thus forming a symmetric or approximatelysymmetric melodic figure such as C4 E4 C4 or C4 E4 C 4)

4 The proximity principle describes a general expectancy for smallrealized intervals

5 The principle of closure depends on pitch direction and intervalsize The degree of closure is higher if there is a change in pitchdirection and if the realized interval is smaller than the implica-

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 11: 2002Povel&Jansen

61Harmonic Factors in Melody Perception

tive interval Closure is also dependent on other style-specificfactors such as duration of the involved tones position in themetric frame and harmony

These variables donrsquot do justice to the comprehensiveness of the implica-tion-realization model as they donrsquot sufficiently reckon with long-term ef-fects and context effects due to harmonic and metrical factors

The principles were quantified and represented using a grid representationby Krumhansl (1995 p 73) and Schellenberg (1996 pp 78ndash79) and testedon three-tone sequences (Cuddy amp Lunney 1995) and on the last three tonesfrom initial fragments of British folk songs atonal melodies and Chinese folksongs (Krumhansl 1995 Schellenberg 1996 1997) by asking listeners to ratehow well the last tone continued the melodic fragments In the latter investiga-tions the first tone of the implicative interval always had a longer duration astronger metrical position and was more stable in the key of the fragmentthan the second tone of the interval These methodological procedures implythat possible effects from metrical and harmonic factors will be minimal To-gether these studies indicate that the principles are indeed valid predictors oflistenersrsquo expectations together with the aforementioned variable ldquotonalityrdquoIt may thus be concluded that the principles that basically concern pitch heightconfigurations play an important role in determining listenersrsquo expectancieswhen perceiving short musical fragments

Both Cuddy and Lunney (1995) and Schellenberg (1996) noted that theoriginal principles differed considerably in their predictive power and pro-posed a modification of the model In the first instance Schellenberg (1996)reduced the model to three principles but later showed that the model couldbe reduced to only two principles without any loss of explanatory power(Schellenberg 1997) These two principles are as follows

1 The proximity principle which states that ldquowhen listeners hearan implicative interval in a melody they expect the next tone tobe proximate to the second tone of the implicative interval (iethey expect a small realized interval)rdquo (Schellenberg 1997 p309) This is the same as saying that a listener when listening toa melody at any time expects a small interval The principle isquantified by assigning a value of 0 to the interval 0 (prime) 1 toa minor second and so on

2 The pitch reversal principle This principle is a linear combina-tion of the (revised) principles of registral direction and registralreturn The two principles are virtually uncorrelated (r = 029)The quantification of the two principles is shown in Figure 4

As the implication-realization theory basically is a bottom-up model thattreats melody primarily as a note-to-note phenomenon as reflected in the

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 12: 2002Povel&Jansen

62 Dirk-Jan Povel amp Erik Jansen

principles just discussed it cannot directly be applied to the perception ofentire sequences Nevertheless we have computed derivations of the afore-mentioned principles applicable to an entire sequence by conceiving a tonesequence as a series of consecutive implication-realization intervals (seeThompson amp Stainton 1998) The exact computation of the two modifiedprinciples called MeanInt and PitchRev is described in the Methods sec-tion

Predictions Based on Specific Sequence Characteristics

Finally we have measured a few attributes of the sequences that mayaffect goodness ratings the number of contour changes the average conso-nance of the intervals in the sequence and the variation in interval sizes

Experiment 1

In this experiment we compare the perception of sequences that (ac-cording to the notions just considered) possess features that interfere withthe generation of a harmonic interpretation with sequences that lack thesefeatures The ldquostandardrdquo sequences contain tones in positions that hampera smooth harmonic interpretation whereas in the ldquoderivedrdquo sequences

Fig 4 Quantification of the principles of pitch proximity and pitch reversal proposed bySchellenberg (1997) Figure reprinted with the kind permission of the author and the Re-gents of the University of California

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 13: 2002Povel&Jansen

63Harmonic Factors in Melody Perception

these tones are replaced or placed in a context that supposedly eliminatesor reduces their interference effect The derived sequences are of three types(1) sequences consisting of arpeggiated chords (2) sequences containingnonchord tones that can be linked to chord tones by using the anchoringmechanism and (3) sequences in which nonchord tones are incorporatedin runs It is predicted that sequences complying with the hypothesizedcharacteristics and that therefore should enable a straightforward harmonicanalysis will yield a higher goodness rating than sequences that lack thesefeatures To obtain a more detailed insight into which tones actually inter-fere with the processing of the sequences participants also rated the indi-vidual tones of the stimuli separately from the global ratings they pro-vided

METHOD

Subjects

Forty-eight listeners participated in the experiment consisting of two groups 22 sub-jects from the Nijmegen university community (median age = 285 years) all of whom wereat least moderately trained in music (mean = 102 years SD = 73 years) and 26 studentsfrom two music cognition classes at Northwestern University (median age = 215 years) allwith a considerable amount of musical training (mean = 152 years SD = 47 years)

Construction of the Stimuli

The 20 stimuli used in the experiment all 10-tone sequences are displayed in Table 1We started with five standard sequences four of which were used in Experiments by CuddyCohen and Mewhort (1981) selected because of the low ratings they had received in thatstudy To each of these sequences 3 tones were added without changing the harmoniccontext so as to make them 10 tones long These standard sequences contain tones thatappear to hamper the creation of a harmonic analysis

From each of these standard sequences three new sequences were derived by alteringone or more tones Three types of transformation were used (1) A transformation yieldinga sequence consisting only of chord tones that is arpeggiated chords denoted harmonic inTable 1 (2) A transformation yielding a sequence containing one or more nonchord tonesthat can be linked to a subsequent chord tone by means of the mechanism of anchoringdenoted anchoring in Table 1 (3) A transformation yielding a sequence containing one ormore nonchord tones that are incorporated in a run (a series of tones with intervals of asemitone or whole tone) denoted run in Table 1 The transformations were achieved byapplying a minimal number of alterations Because we wanted to avoid repetition of thesame stimulus and because in the anchoring and run transformations all nonchord toneshad to be accommodated by these principles in a number of cases the derived sequencesdiffer quite a bit from the standard sequences However for the question under consider-ation this difference is of no consequence

To control for the temporal grouping or segmentation all sequences were presentedwith a distinct timing and dynamic pattern that induced a triple grouping such that the firstand the last tone fell on a downbeat This timing and accentuation pattern was obtained byrecording the performance of a 10-tone sequence presented in a 98 meter The averageinteronset interval between the tones of this template was 600 ms whereas the velocity(intensity) of the first tones of the groups (downbeats) was approximately 60 (on a scalefrom 1to 127) that of the second tone 38 and that of the last tone 33 All stimuli weregenerated with this timing and accentuation pattern

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 14: 2002Povel&Jansen

64 Dirk-Jan Povel amp Erik Jansen

Stimulus Presentation

The sequences were presented in a different random order for each of the Nijmegenparticipants The students from Northwestern University received only two random ordersbecause the experiment was run in two group sessions Each sequence was played at arandomly determined pitch height varying between 6 semitones above and below their no-tated pitch in Table 1 The sequences were played through a Yamaha PSR-620 synthesizerusing the Grand Piano sound Both stimulus presentation and response collection werecontrolled by means of a program written in REALbasic running on a Macintosh G3 com-puter

Procedure

The participants from the Nijmegen group performed the experiment individually in thefollowing way A participant was seated in front of a computer screen that displayed awindow with buttons as shown in Figure 5 In the middle of the window the stimulus isshown as a sequence divided into three groups of three tones followed by one single toneInitially only the button in the corner right below marked ldquostartrdquo was enabled When thisbutton was pressed its text changed into ldquonextrdquo and the ldquoallrdquo button was enabled Upon

TABLE 1Tone Sequences Used in Experiment 1

No Type Sequence

1 Standarda C5 E5 G5 E5 C5 G5 F5 D5 A4 C52 Harmonic C

5 E

5 G

5 F

5 D

5 A

4 D

5 B

4 G

4 C

53 Anchoring C

5 E

5 G

5 F

5 D

5 A

4 B

4 G

4 D

5 C

54 Run C5 E5 G5 F5 D5 A4 B4 C5 D5 C55 Standardb C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

56 Harmonic C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

57 Anchoring C5 E5 G5 F5 D 5 D5 B4 G4 B4 C58 Run C

5 E

5 G

5 F

5 D

5 D

5 C

5 B

4 D

5 C

59 Standard C

5 E

5 G

5 E

5 G

5 C

5 F

5 D

5 B

4 C

510 Harmonic C5 E5 G5 E5 G5 C5 F 5 D 5 B4 E511 Anchoring C

5 D

5 E

5 C

5 F

5 G

5 B

4 D

5 B

4 G

412 Run C

5 D

5 D

5 E

5 F

5 F

5 G

5 D

5 B

4 G

513 Standardc C5 E5 G5 E5 C5 G5 F 5 D5 B4 C514 Harmonic C

5 E

5 G

5 E

5 C

5 G

5 F

5 D

5 G

4 C

515 Anchoring C

5 E

5 G

5 E

5 C

5 F

5 G

5 D

5 B

4 G

416 Run C5 E5 G5 F 5 F5 D5 B4 G4 B4 C517 Standardd B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 A

4 C

518 Harmonic B

4 E

5 G

5 E

5 B

4 G

5 F

5 D

5 B

4 E

519 Anchoring B4 E5 G5 F5 D5 A 4 A4 C 5 E5 D520 Run B

4 E

5 G

5 F

5 D

5 A

4 A

4 B

4 C

5 D

5

NOTEmdashHarmonic = sequences inducing a common chord progression Anchoring = se-quence contains nonchord tones that are captured by means of anchoring Run = sequencecontains nonchord tones that are captured in a run (All chromatic notes are notated assharps)

aDerived from Cuddy et al (1981) Expt 2 S21bDerived from Cuddy et al (1981) Expt 2 S41cDerived from Cuddy et al (1981) Expt 1 13dDerived from Cuddy et al (1981) Expt 1 24

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 15: 2002Povel&Jansen

65Harmonic Factors in Melody Perception

pressing the ldquoallrdquo button the sequence was presented Next the five radio buttons at the topof the window flanked with the texts ldquobadrdquo on the left and ldquogoodrdquo on the right and the 10pop-up menus below the notes were activated The participant rated the global goodness orwellformedness of the sequence by activating one of the radio buttons on top of the screenNext in case the global rating was lower than 5 the subject indicated which tone or whichtones sounded bad using the pop-up menus which served as five-point rating scales for theindividual tones In doing these tasks participants could listen to the stimulus as manytimes as desired The Northwestern students heard each sequence twice and provided theirratings on response forms

RESULTS

All participants enjoyed the experiment and experienced the tasks asinteresting musically relevant and not particularly difficult When askedwhat criteria they used to rate the sequence participants gave answers likeldquoI tried to determine whether any of the tones were not in the keyrdquo ldquoI triedto determine whether or not the tones in the sequence were actually re-solvedrdquo ldquoI paid attention to continuity in the phraserdquo ldquoVery out of placetones dissonant leaps unlikely continuations made the sequence soundless goodrdquo Participants mentioned that a sequence that sounded ratherbad at first presentation often sounded better upon second hearing

Some participants reported that a particularly bad tone (a ldquojarringrdquo note)tends to influence the subsequent tones in the sense that they also appear tosound bad More precisely the listeners often found it difficult to tell whether

Fig 5 Layout of the computer window used in the experiments The window displays thetemporal configuration of the tone sequence a five-point rating scale for judging the globalgoodness of the sequence and the pop-up menus below the tones which serve as ratingscales for the individual tones In Experiment 2 the tone sequence contained one extragroup of three tones

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 16: 2002Povel&Jansen

66 Dirk-Jan Povel amp Erik Jansen

the tones following a jarring tone were actually good or bad that is theycould not tell whether these tones fit with the fragment that preceded thejarring tone We shall come back to this in the Discussion

Analysis of the Global Ratings

The mean global ratings for the stimuli are shown in Table 2 The meanglobal ratings for the four conditions (types of sequences) standard har-monic anchoring and run averaged over all participants are 2612 42383375 and 3367 respectively Figure 6 displays the results on the fourconditions for the two groups of subjects It indicates that although theNorthwestern studentsrsquo ratings for the last three conditions are somewhatlower the overall pattern is the same the standard sequences were ratedlowest the harmonic sequences highest and the sequences in the anchor-ing and run conditions obtained a rating in between these two values Ananalysis of variance (within-subject design) performed on the mean ratingsof the four types of stimuli shows a significant effect for conditions F(345) = 8858 p lt 0001

To determine the significance of the differences between the individualgroups planned comparisons were performed T-tests after Bonferroniadjustment (alpha = 008) show for both groups of participants signifi-cant differences for all six pairs of conditions except for the differencebetween the anchoring and run condition

1 2 3 40

1

2

3

4

5NijmegenEvanston

Conditions

Mea

n ra

tings

Fig 6 Mean global ratings of the two groups of participants for the four categories ofstimuli in Experiment 1 (1) Standard (2) Harmonic (3) Anchoring and (4) Run

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 17: 2002Povel&Jansen

67Harmonic Factors in Melody Perception

Analysis of the Individual Tone Ratings

To compare the ratings of the individual tones with the ratings providedfor the sequences as a whole (the global ratings) we calculated the meantone rating per sequence by averaging the ratings over tones and subjectsThe correlation between the mean tone ratings and the global ratings is962 (p lt 001) indicating that the judgment of the individual tones is wellreflected in the global ratings

Figure 7 shows the ratings of the individual notes for all 20 stimuliThe four rows represent the four conditions standard harmonic an-choring and run each containing five sequences A few typical charac-teristics of these response profiles should be mentioned First in sev-eral cases a tone that is rated low in the standard condition (first row)is also rated relatively low in the other conditions Such is the case forthe A in the first column for the D in the second column for the F inthe fourth column and the A in the last column Note however thatin at least one case the chromatic tone is rated just as high as the dia-tonic tones (Sequence 12) Second note the differing ratings of the fivesequences in the harmonic condition The highest ratings are given toSequences 6 14 and 18 with an underlying IndashVndashI progression (18 is aminor mode) an intermediate rating to Sequence 2 with an underlyingIndashiindashVndashI progression and the lowest rating to Sequence 10 which in-duces a VIndashVndashi progression

Fig 7 Mean ratings of the individual tones in Experiment 1 All subjects (N = 48)

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 18: 2002Povel&Jansen

68 Dirk-Jan Povel amp Erik Jansen

Comparison with Alternative Explanations

As mentioned in the Introduction we have compared the explanationjust advanced with three alternative explanations one based on the tonalhierarchy concept of Krumhansl (1990) one on the formalization of partof the implication-realization model of Narmour (1990) by Schellenberg(1997) and one on the presence of specific features in the sequence

To test how well the average position of the tones in the tonal hierarchy(stability) explains the ratings collected in the experiment we have deter-mined the stability of each separate tone in a sequence and calculated themean of these stabilities represented in the variable MeanStab The tonalhierarchy used in the computations is that of the key of C major for Stimuli1-16 E minor for Stimuli 17 and 18 and D major for Stimuli 19 and 20

To test the predictions from Schellenbergrsquos reduction of the implicationndashrealization model of Narmour we based ourselves on the two principlespitch proximity and pitch reversal shown by Schellenberg (1997) to havethe same explanatory power as the original principles and have modifiedthese variables such that they are applicable to an entire sequence in thefollowing way

1 To obtain a global measure of the pitch proximity principleapplicable to an entire melody we have computed the mean ofthe absolute interval sizes called MeanInt According to the pitchproximity principle the first interval should not be includedbecause the principle applies to two consecutive intervals butthis seems to be too dogmatic as the first interval will normallyalso be judged Moreover it would not make any difference forour stimuli because they all begin with the same two intervals

2 To obtain a global measure of the pitch reversal principle ap-plicable to an entire melody we have computed the pitch rever-sal implication for all successive pairs of intervals in a sequenceusing the quantification of the principle as displayed in Figure 4Next we have calculated the mean pitch reversal implication bysumming the individual values after adding + 1 to each value (tocompensate for the fact that the local value may be ndash1 see Figure4) As the tritone (interval of 6 semitones) is not dealt with in theprinciple we have treated it in the same way as an interval of 7semitones This variable is called PitchRev

To test the possible effect from specific features in the sequences we havedetermined three alternative variables the number of contour changesContChanges the average consonance value of the intervals in the se-quencemdashbased on the study of Malmberg (1918) as quantified byKrumhansl (1990 p 57)mdashConsDis and the variation of the interval sizes

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 19: 2002Povel&Jansen

69Harmonic Factors in Melody Perception

in the sequence VarInt (defined as the standard deviation of the intervalsizes) The values of the variables for the 20 stimuli is shown in Table 2

To compare the predictive power of the alternative explanations withour explanation we first calculated the Pearson correlation among the in-dependent variables and the dependent variable (Table 3) Inspection of thecorrelation table reveals that only the variable Category is significantlycorrelated (r = -69) with the dependent variable Rating and that several ofthe independent variables are highly intercorrelated Although all othervariables are not significantly correlated with Rating we performed a hier-archical multiple regression in which the predictors associated with thefour explanations are entered incrementally in the analysis (Tabachnik ampFidell 2001 p 133 ff) Thus in the first step the variable MeanStab basedon the tonal hierarchy notion was entered in the next step the variablesMeanInt and PitchRev based on the implication-realization model wereadded in the third step the variables ContChan ConsDis and VarInt wereadded and in the final step the variable Category related to our explana-tion was added The results of the analysis are shown in Table 4

TABLE 2Values of the Dependent and Independent Variables for the Stimuli Used

in Experiment 1

Stim No Rating Category MeanStab MeanInt PitchRev ContChanges ConsDis VarInt

1 306 4 494 378 15 4 559 14 2 394 1 451 378 175 4 532 103 3 421 2 450 356 15 5 624 157 4 392 3 462 267 1375 3 78 115 5 258 4 475 356 1687 6 612 164 6 492 1 486 356 2062 6 601 157 7 285 2 431 267 1375 2 764 115 8 308 3 449 222 1562 3 871 092 9 267 4 459 378 1812 6 637 16910 296 1 440 422 2 6 546 1411 335 2 415 367 1375 5 738 21112 394 3 399 256 1 2 912 23113 294 4 471 333 15 4 649 1714 458 1 509 422 1625 4 509 16915 346 2 459 367 125 3 627 13316 321 3 433 267 1375 2 751 12517 181 4 360 4 15 4 591 1718 479 1 367 411 15 4 572 18519 3 2 402 3 1375 3 694 11520 269 3 404 256 1375 2 8 126

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 20: 2002Povel&Jansen

70 Dirk-Jan Povel amp Erik Jansen

The results of the regression analyses confirm the picture already pre-sented by the correlations the predictors related to the alternative explana-tions contribute only a very small and nonsignificant part to the explainedvariance whereas the variable Category does explain a significant and sub-stantive portion of the variance in the data thereby corroborating the re-sults of the analysis of variance reported earlier

DISCUSSION

In this experiment we have shown that ldquobadrdquo sequences (sequences re-ceiving low goodness ratings) can be transformed into ldquogoodrdquo sequenceseither by altering the alleged bad tones such that a harmonic analysis be-

TABLE 3Correlations Between the Dependent and Independent Variables of

Experiment 1Global Category Mean Mean Pitch Cons VarInt ContRatings Stab Int Rev Dis Changes

Global ratings 1Category -069 1MeanStab 025 001 1MeanInt 016 -030 014 1PitchRev 011 -028 041 052 1ConsDis -017 031 -031 -091 -062 1VarInt 018 -009 -021 035 018 -003 1ContChanges 008 -010 034 070 079 -063 029 1

p lt 05 p lt 01

TABLE 4Summary of a Hierarchical Regression Analysis on the Data of

Experiment 1

Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 061 008 8080 061 1162 (1 18) 2952 082 -090 8471 022 189 (2 16) 8303 447 -170 8776 117 635 (313) 6054 735 580 5256 535 24247 (1 12) 000

NOTEmdash Table shows the variance explained (R2 change) by the three alternative explana-tions (see text) and by the harmonic model proposed in this study The predictors added ateach step are printed in bold Model 1 Predictors MeanStab Model 2 Predictors MeanStabMeanInt PitchRev Model 3 Predictors MeanStab MeanInt PitchRev ContChangesConsDis VarInt Model 4 Predictors MeanStab MeanInt PitchRev ContChanges ConsDisVarint Category

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 21: 2002Povel&Jansen

71Harmonic Factors in Melody Perception

comes feasible or by altering tones in the neighborhood of an alleged badtone such that it can be anchored to a chord tone or incorporated in a runThese results support the basic idea that initiated this study namely that inorder to perceive a tone sequence as a tonal melody the listener must suc-ceed in creating a harmonic analysis If this analysis is hampered by theoccurrence of nonchord tones the sequence will receive a low rating Inaddition it was shown that the interference effect of nonchord tones is lessif they are somehow assimilated by other tones in the sequence

As regards the specific predictions made we may conclude that

1 Sequences that allow a straightforward harmonic analysis yielda higher goodness rating than sequences that do not allow this(condition harmonic vs the three other conditions) The datafurther suggest that a sequence is rated higher if the induced har-monic progression is more common

2 Sequences containing nonchord tones that are somehow assimi-lated either by means of the mechanism of anchoring or by be-ing assimilated in a run are rated higher than sequences that donot accommodate nonchord tones in these ways (condition stan-dard vs conditions anchoring and run) In the anchoring andrun conditions the nonchord tones are rated lower than the chordtones This important finding indicates that in general nonchordtones even when anchored or being part of a run are still expe-rienced as having some interference effect The results also indi-cate that anchoring is not an all or none process but that itsoperation depends on the actual tones involved For instanceSequence 12 in which the F is followed by a G is rated higherthan Sequence 16 in which the F is followed by F

In addition we may conclude that the goodness rating of the individualtones is a valuable response indicator because it provides a fairly detailedpicture of which tones are difficult to incorporate in a musical interpreta-tion

Two other observations made while conducting this study are worthmentioning First the finding that a series often sounds better when heardfor the second time may indicate that the process of perception is a rela-tively slow and dynamic process in which the listener tries to make sense ofthe series of tones by developing different hypotheses about how to incor-porate the tones in a coherent percept Second the individual tone ratingsshown in Figure 7 indicate that often not only the jarring tone but also thesubsequent tones receive a lower rating One participant expressed thisvery clearly ldquoI donrsquot seem to be able to interpret the tones that follow thebad tone I cannot decide whether or not they fit in the sequence as a wholerdquoThis result suggests that it is not possible to relate tones to previous tones if

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 22: 2002Povel&Jansen

72 Dirk-Jan Povel amp Erik Jansen

the harmonic analysis is interrupted and demonstrates the critical signifi-cance of an effective harmonic interpretation in music perception

A discussion of the comparison between the predictions of our modelwith predictions based on the other explanations is postponed until wereport a similar analysis on the data of the next experiment

Because in this experiment only a limited number of sequences was usedand because the potential variables were not varied systematically it isdifficult to tell to what extent these findings can be generalized to othersequences Besides we would like to establish whether there exist alterna-tive mechanisms that reduce the interference effect of nonchord tones Theseissues are further examined in Experiment 2

Experiment 2

In the previous experiment we have provided some evidence that good-ness ratings of tone sequences depend on whether the listener is able tocreate a harmonic analysis and that the positioning and context of nonchordtones are critical in determining whether such an analysis will succeed Inthis experiment we examine the conditions that play a role in the induc-tion of an uninterrupted harmonic analysis in a more systematic way byusing an extended set of stimuli in which all possible nonchord tones areintroduced For this purpose we started with a prototypical sequence basedon a IndashVndashI progression from which a number of new sequences were de-rived by replacing one tone in the V section by another tone within therange of an octave With this set of stimuli we want to verify the findingsof Experiment 1 and to discover whether there are additional factors thatinteract with the factors discerned so far

METHOD

Subjects

Twenty-one adult listeners all from the Nijmegen university community participated inthe experiment (median age = 27 years) All participants were at least moderately trained inmusic with minimally 5 years of lessons on a musical instrument or in singing (mean = 111years SD = 59 years)

Construction of the Stimuli

Beginning with the prototypical sequence C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 basedon a IndashVndashI progression we constructed a set of sequences by substituting the tone F5 foreach of the tones between G

4 and A

5 (a fifth below and above the tone following F

5) From

this set we eliminated the sequences that contained two consecutive identical tones (theimmediate repetition of a tone does not introduce any extra harmonic factor whereas it wasfound in a pilot experiment that a sequence containing a repeated tone is very conspicuousand difficult to judge in relation to sequences not containing a repetition) In a similar

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 23: 2002Povel&Jansen

73Harmonic Factors in Melody Perception

fashion the last D5 and the last B4 of the sequence were substituted thus obtaining the set of35 stimuli displayed in Table 5 In the set all possible nonchord tones were introduced atthree different positions in the V segment of the sequences

To control the metrical interpretation the stimuli were generated with timing and dy-namic features obtained from an actual performance of the prototypical stimulus played ina 128 meter that is very similar to the metrical configuration of Experiment 1 except thatthis sequence has four instead of three groups of three tones The average interonset intervalbetween the tones of the metrical template was 600 ms whereas the velocity (intensity) of

TABLE 5Stimuli Used in Experiment 2

Mean RatingNo Sequence Rating Subst

1 G 4 219 2 2 A4 324 333 3 A

4305 281

4 B4 443 457 5 C5 367 376 6 C

5233 229

7 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5 D5 B4 C5 267 248 8 E5 338 338 9 F

55 5

10 F 5 267 27111 G 5 281 25712 A

5376 395

13 E4 19 18114 F 4 157 15715 G

4467 476

16 G 4 243 22917 A4 457 46718 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5F

5A

4B

4C

53 31

19 C5 333 34820 C 5 233 20521 D

5257 238

22 E5 4 42423 F 5 257 21424 F

4248 31

25 F 4 186 17126 G4 452 44827 G

4181 176

28 A4 295 29529 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 233 21930 C

5248 219

31 D 5 319 28132 E5 467 4933 F

5386 367

34 F 5 205 17135 G5 462 486

NOTEmdashTo clarify the construction of the stimuli the complete sequence is shown onlythree times once for each position in which tones are substituted Penultimate columnmean global rating Last column mean rating of substituted tone Stimulus 9 is the proto-typical sequence from which all other sequences are derived

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 24: 2002Povel&Jansen

74 Dirk-Jan Povel amp Erik Jansen

the first tones of the groups (downbeats) was approximately 60 (on a scale from 1 to 127)that of the second tone 38 and that of the last tone 33 All stimuli were generated with thistiming and velocity pattern

Stimulus Presentation and Procedure

Stimulus presentation and procedure were the same as in Experiment 1 except that thewindow on the computer was adapted for the longer experimental stimuli and the subjectrsquostask was to judge whether the sequence might occur in a simple piece of tonal music As inthe previous experiment participants indicated which tone(s) did not seem to fit well in thesequence by using the five-point rating scales below each tone To get familiarized with theprocedure the participants practiced with six stimuli not used in the experiment

RESULTS

The participants liked the musical relevance of the task which they foundrelatively easy to perform Some participants remarked that sometimes itwas not so much the substituted tone that was wrong but rather the waythe sequence continued after that tone

Between-subject reliability is quite high as witnessed by a split-half cor-relation of 94 Correlation between the mean global ratings and the meanof the individual tone ratings is very high 973 as in the preceding experi-ment Also the correlation between the global ratings and the ratings of thesubstituted tone is very high 984 indicating that the quality of the se-quences is mainly determined by the substituted tone

The mean global ratings of the stimuli and the rating of the substitutedtone are shown in Table 5 In Table 6 the stimuli are annotated and or-dered according to rating

Inspection of Table 6 shows that sequences containing only chord tonesare rated highest those containing a local nonchord tone are rated lowerand those containing a nonkey tone are rated lowest More importantlythe table also shows that sequences containing a nonchord tone (eitherdiatonic or chromatic) which can be accommodated either in a harmonicinterpretation or by means of anchoring are rated higher than those thatcannot be captured by one of these mechanisms To make this apparent inColumn 3 of Table 6 we have indicated whether a nonchord tone can beaccommodated in an alternative less likely harmonic analysis (denotedwith the abbreviation AltHarm) or may be resolved by means of anchoring(denoted Anch) Next we have distinguished three conditions by dividingthe sequences into three categories labeled 1 2 and 3 in the last column(1) sequences that induce a IndashVndashI progression (6 sequences) (2) sequencesin which a nonchord tone is captured either by an alternative harmoniza-tion or by anchoring (19 sequences) (3) sequences containing a nonchordtone that cannot be captured by either of the two mechanisms (10 sequences)The mean ratings for the stimuli in these three conditions are respectively

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 25: 2002Povel&Jansen

75Harmonic Factors in Melody Perception

TABLE 6Classification and Ratings of the Stimuli in Experiment 2

Ordered by Rating

No Sequence Type Glob Subst C

9 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 B4 C5 H (prototypical 5 5 1 sequence)

32 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 E

5 C

5H 467 476 1

15 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 G4 B4 C5 H 467 49 135 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G5 C5 H 462 486 117 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 A

4 B

4 C

5AltHarm 457 467 2 IV-V-IRun

26 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 G4 C5 H 452 448 14 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 B

4 D

5 B

4 C

5H 443 457 1

22 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5-E5 B4 C5 Anch 4 424 233 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F5 C5 AltHarm ii-I 386 367 212 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

5 D

5 B

4 C

5AltHarm ii-V-I 376 395 2

5 C5 E5 C5 G5 E5 C5 B4 D5 G5 C5-D5 B4 C5 Anch 367 376 28 C5 E5 C5 G5 E5 C5 B4 D5 G5 E5-D5 B4 C5 Anch 338 338 219 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 C

5-B

4 C

5AnchAltHarm 333 348 2 IV-V-I

2 C5 E5 C5 G5 E5 C5 B4 D5 G5 A4-D5-B4 C5 DelAnchAlt 324 333 2 Harm ii-V

31 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 D 5 C5 AltHarm V-i 319 281 2 (minor)

3 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 A

4-D

5-B

4 C

5DelAnchRun 305 281 2

18 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 A 4-B4 C5 Anch 3 31 228 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A4 C5 AltHarm ii-I 295 295 211 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 G

5 D

5 B

4 C

5AltHarm III-I 281 257 2

10 C5 E5 C5 G5 E5 C5 B4 D5 G5 F 5 D5 B4 C5 - 267 248 37 C5 E5 C5 G5 E5 C5 B4 D5 G5 D 5-D5 B4 C5 Anch 267 271 223 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 F

5 B

4 C

5- 257 238 3

21 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D 5 B4 C5 AltHarm V-I 257 214 2 (minor)

30 C5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 C

5 C

5AltHarm ii-I 248 31 2 Run

24 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F4 C5 AltHarm ii-I 248 219 216 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 G

4 B

4 C

5- 243 229 3

29 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 A 4 C5 - 233 229 320 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 C 5-B4-C5 AnchDelAnch 233 205 2

(C-C)6 C5 E5 C5 G5 E5 C5 B4 D5 G5 C 5-D5 B4 C5 Anch 233 219 21 C5 E5 C5 G5 E5 C5 B4 D5 G5 G 4 D5 B4 C5 - 219 2 334 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 F

5 C

5- 205 171 3

13 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 E4 B4 C5 - 19 181 325 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 D5 F 4 C5 - 186 171 327 C

5 E

5 C

5 G

5 E

5 C

5 B

4 D

5 G

5 F

5 D

5 G

4 C

5- 181 176 3

14 C5 E5 C5 G5 E5 C5 B4 D5 G5 F5 F 4 B4 C5 - 157 157 3

NOTEmdashGlob = global rating Subst = rating of the substituted tone H = harmonic Anch= Anchoring (indicated with a dash between the tones) DelAnch = delayed anchoringAltHarm = alternative harmonic analysis - = does not allow an harmonic analysis C =condition

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 26: 2002Povel&Jansen

76 Dirk-Jan Povel amp Erik Jansen

465 314 and 214 (Figure 8) An analysis of variance performed on thesubjectsrsquo mean ratings of the three conditions exhibits a highly significanteffect for conditions F(2 19) = 9110 p lt 0001 Planned comparisonsapplying Bonferronirsquos adjustment (alpha = 016) indicate that all threepairwise comparisons are significantly different (all p values lt 0001)

To examine whether the position of the substituted tone affected theratings we performed a two-way analysis of variance with Condition (3levels) and Position (3 levels first second or third tone in penultimatebeat) as within factors This analysis shows a significant effect for Condi-tion (as in the former analysis) whereas the effect of Position and the inter-action between the factors are not significant

Comparison with Alternative Explanations

Similar to the first experiment we have compared our explanation ofthe data with the three alternative explanations Table 7 shows the valuesof the dependent and independent variables for the 35 stimuli and Table 8presents the intercorrelations between the independent and dependent vari-ables

It appears that two variables correlate significantly with the dependentvariable rating namely Category (r = -85) and MeanStab (r = 528) indi-cating that these factors on their own (without taking into account correla-

1 2 30

1

2

3

4

5

Conditions

Mea

n ra

ting

Fig 8 Mean global ratings for the three conditions in Experiment 2 Condition 1 containssequences clearly inducing a IndashVndashI progression Condition 2 contains sequences in which anonchord tone is assimilated by anchoring or by an alternative harmonization Condition 3contains sequences containing a nonchord tone that cannot be captured See text for expla-nation

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 27: 2002Povel&Jansen

77Harmonic Factors in Melody Perception

tions with other variables) respectively explain 722 and 279 of thevariance in the data The correlations of the other variables with the depen-dent variable are very low and statistically not significant To assess therelative contribution of the variables Category and MeanStab we performedtwo Hierarchical Multiple Regressions one in which the variable MeanStab

TABLE 7Values of the Dependent and Independent Variables

for the Stimuli Used in Experiment 2

Mean MeanStNo Rating Category Stab Mean Int Pitch Rev Cont Chan Cons Dis Var Int

1 219 3 459 433 168 8 669 262 2 324 2 469 417 173 8 612 237 3 305 2 458 4 173 8 592 216 4 443 1 463 383 173 8 621 199 5 367 2 489 367 186 8 609 189 6 233 2 458 35 186 8 675 185 7 267 2 458 333 164 6 645 17 8 338 2 474 333 164 6 647 16 9 5 1 472 333 15 6 647 1610 267 3 46 333 15 6 645 1711 281 2 459 35 141 6 675 18512 376 2 469 367 141 6 609 18913 19 3 479 45 136 6 648 31814 157 3 465 417 145 6 65 26415 467 1 485 4 145 6 649 24216 243 3 464 383 145 6 632 22317 457 2 473 367 145 6 669 20918 3 2 463 35 145 6 648 20219 333 2 494 333 15 6 656 1820 233 2 462 333 15 6 654 16521 257 2 463 333 136 6 654 16522 4 2 479 333 136 6 656 1823 257 3 465 35 159 8 648 20224 248 2 481 433 159 6 555 22125 186 3 469 417 159 6 62 19526 452 1 49 4 159 6 54 17327 181 3 468 383 145 6 605 15728 295 2 478 367 15 6 578 14929 233 3 467 35 15 6 615 1530 248 2 467 317 136 5 686 17231 319 2 468 333 164 7 646 1632 467 1 483 35 164 7 615 1533 386 2 481 367 164 7 578 14934 205 3 469 383 164 7 605 15735 462 1 49 4 164 7 54 173

NOTEmdashMeanStab = mean stability of the notes in the sequence MeanInt = mean intervalof the notes (proximity principle) PitchRev = expectancy based on the pitch reversal prin-ciple ContChanges = number of contour changes ConsDis = mean consonance of the inter-vals VarInt = standard deviation of the interval sizes

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 28: 2002Povel&Jansen

78 Dirk-Jan Povel amp Erik Jansen

was entered in the first step and the variable Category in the second stepand another one using the reverse entry order The results of these analysesare shown in Table 9

The analyses reveal that the variable Category adds 477 to the vari-ance explained (R2 change) if the variable MeanStab is entered first (Analysis1) whereas the variable MeanStab adds only 33 to the explained vari-ance if the variable Category is entered first (Analysis 2) From this it maybe concluded that although MeanStab does explain part of the data virtu-ally all of it is also explained by Category Apparently the variablesMeanStab and Category (or better the underlying models) both describeaspects of the data that are relevant in explaining the responses

TABLE 8Correlations Between the Dependent and Independent Variables of

Experiment 2

Rating Category Mean Mean Pitch ContStab Int Rev Changes ConsDis Var Int

Rating 1Category -085 1MeanStab 053 -043 1Mean Int -018 019 016 1PitchRev 012 -017 001 017 1ContChanges 010 -005 -012 025 082 1ConsDis -024 025 -049 -042 -030 -015 1VarInt -024 027 -006 072 -008 015 020 1

p lt 05 p lt 01

TABLE 9Summary of Two Hierarchical Regression Analyses Performed on the

Data of Experiment 2

ANALYSIS 1 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 280 258 8365 280 12816 (1 33) 0012 756 741 4943 476 62492 (1 32) 000

ANALYSIS 2 Standard Change Statistics

Model R2 R2 Adjusted Error R2 Change F Change p

1 723 715 5186 723 86211 (1 33) 0002 756 741 4943 033 4315 (1 32) 046

NOTEmdashIn Analysis 1 the variable MeanStab is entered in the first step and the variableCategory added in the second In Analysis 2 the ordering of entering is reversed

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 29: 2002Povel&Jansen

79Harmonic Factors in Melody Perception

DISCUSSION

In this experiment we used an extended set of stimuli to verify the hy-pothesis that to perceive a sequence of tones as a melody the listener mustsucceed in making a harmonic analysis If we compare the mean rating ofthe stimuli in Condition 1 containing sequences that strongly induce a I-V-I progression (465) with that of the sequences in Condition 3 for whichthe formation of a complete progression seems impossible (214) we mayconclude that the data support this hypothesis

The second purpose of the experiment was to discover the conditionsthat enable a harmonic analysis in spite of the occurrence of a nonchordtone It was hypothesized that listeners either attempt to make an alterna-tive harmonic analysis that transforms the nonchord tone into a chord toneor try to anchor the nonchord tone to a subsequent chord tone To differ-ent extents these conditions are fulfilled in the sequences in Condition 2which on average were indeed rated higher than the sequences in Condi-tion 3 (mean rating 314 vs 214) Thus the data indicate that the degree towhich nonchord tones interfere with the formation of a complete harmonicanalysis strongly depends on the context in which these nonchord tones ap-pear This context is rather local when the mechanism of anchoring is appliedbut more global in cases where the nonchord tone is transformed into a chordtone by assigning an alternative harmonic analysis Although the sequences inCondition 2 are rated higher than those in Condition 3 they are neverthelessrated lower than the sequences in Condition 1 This may be because in Condi-tion 2 a harmonic analysis cannot be done as readily as in Condition 1 requir-ing additional processing to arrive at a musical interpretation

The sequences in Condition 2 represent a rather extensive class of stimulithat are supposed to be processed in different ways to arrive at a harmonicanalysis either by means of anchoring or by means of developing a lesscommon harmonic analysis As shown in Table 6 for some of the stimulithere is theoretically more than one solution to the problem and it is notclear which of those solutions is actually applied by the listeners It is con-ceivable that these sequences are processed in both ways and that theirresults are maintained as alternative codes reflecting the basic ambiguityof musical interpretation Anyway it is clear that the details of this pro-cessing are still not well understood and must be further examined Asregards the mechanism of anchoring the data imply that it is not an all-or-none process chromatic tones are generally less well anchored than dia-tonic nonchord tones With respect to the suggested alternative harmonicanalyses the data indicate that the less common the harmonic progressionthe lower the rating compare for instance stimuli 17 (IndashIVndashVndashI rating457) and 24 (IndashVndashiindashI rating 248) As we donrsquot have sufficient data toexamine this properly we cannot go into this any further

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 30: 2002Povel&Jansen

80 Dirk-Jan Povel amp Erik Jansen

The last goal of this experiment was to examine whether there are otherconditions besides anchoring that may facilitate the incorporation of anonchord tone in a harmonic analysis Mainly for that purpose we intro-duced all tones within an octave in the stimuli On the basis of the analysisof the stimuli in Table 6 and the successful prediction of the responsesbased on that analysis we believe that there are no other alternative mecha-nisms that enable the assimilation of nonchord tones

The comparison between the explanation of the data based on the mecha-nisms proposed in this study and those based on some alternative explana-tions indicates that (a) the variable MeanStab associated with Krumhanslrsquostonal hierarchy concept was found to explain part of the variance but thefactor did not explain any variance not already explained by our modelThese findings are understandable as MeanStab is correlated with Cat-egory (r = 43) and thus apparently measures an attribute that is related toharmonic interpretation Indeed as MeanStab is related to the averageposition of the tones in the key hierarchy it will be higher if the sequencedoes not contain a nonkey tone and lower if it does (especially since mostof the nonchord tones in the stimulus set are nondiatonic tones) (b) thepredictions based on the principles derived from Narmourrsquos implication-realization model do not come out right (c) the variables number of con-tour changes average consonance and variability of the interval sizes donot contribute to explaining the variance in the data We return to thisissue later

General Discussion

In two experiments listeners judged the melodic goodness of tone se-quences differing in the extent in which they at least theoretically induce aharmonic interpretation The experiments yielded two results (1) they re-vealed the factors that play a role in making a harmonic interpretation (2)they provided experimental evidence that the goodness ratings can be ex-plained by the effort needed to arrive at a harmonic analysis as well as bythe type of harmonic analysis effectively generated as specified later

Two global classes of single (unaccompanied) tone sequences were usedin the study (a) sequences whose constituents can be conceived as a succes-sion of arpeggiated chords that is once a harmonic interpretation has beenmade all tones are elements of one of the chords and (b) sequences forwhich (a) does not hold that is after a harmonic interpretation has beenmade there are tones left that are not part of one of the chords usuallycalled nonchord tones Comparing the goodness judgments of these twoclasses of sequences it is found that the former receive a much higher rat-ing than the latter Furthermore ratings of more common progressions of

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 31: 2002Povel&Jansen

81Harmonic Factors in Melody Perception

chords (eg IndashVndashI) are higher than those of less common chord progres-sions (eg IndashiindashI)

Next those tone sequences containing nonchord tones were divided intothree subclasses (a) sequences containing nonchord tones that can be an-chored to a subsequent chord tone (b) sequences in which the nonchordtones are part of a run (c) sequences for which the conditions under (a)and (b) do not apply The ratings of the sequences in subclasses (a) and (b)are significantly higher than those for the sequences in subclass (c) Ratingsof the sequences in classes (a) and (b) did not differ It is interesting that thesequences for which the nonchord tones can be anchored or are part of arun were in contrast to what we had expected not rated as high as thosethat do not contain nonchord tones These lower ratings may reflect theadditional effort needed to assimilate these tones in the development of aharmonic interpretation

The results of this study can be summarized as follows (a) the formationof a harmonic interpretation is of paramount importance in the processingof tonal music and largely determines the goodness ratings of these se-quences (b) nonchord tones may seriously hamper the development of aharmonic analysis (c) the interference effect of nonchord tones depends ontheir context and is lessened if the nonchord tones can be assimilated ei-ther by means of anchoring or by being conceived as part of a run Theseresults corroborate those of Cuddy et al (1981) by showing that percep-tual responses to tone sequences are greatly determined by the tonal struc-ture of the sequences In addition they reveal the conditions in whichnonchord tones are ldquoassimilatedrdquo to chord tones and thus do not hamperthe creation of a harmonic interpretation

The incremental processes that give rise to a harmonic interpretationmay now tentatively be described as follows After having heard the firstfew tones of the sequence the listener attempts to detect the underlyingchord This chord is then assumed to be a tonic chord thus yielding apreliminary key candidate With the (provisional) key is associated a wholeset of expectations pertaining to the occurrence of tones (eg diatonic tonesare more likely than chromatic tones) and chords (chords on the first fourthand fifth scale degree are more likely than chords on the other degrees)and so on These expectations also include expectations regarding the or-der of tones and chords For instance if the listener has recognized a seriesof tones as a dominant seventh chord an expectation for elements of thetonic chord or less likely for elements of the chord on the sixth degree willbe created If these expectations are fulfilled the creation of a harmonicinterpretation will succeed and the tone sequence will be perceived as anacceptable musical event

As we have seen the main difficulty in developing a harmonic interpre-tation is the occurrence of nonchord tones tones that do not fit in the

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 32: 2002Povel&Jansen

82 Dirk-Jan Povel amp Erik Jansen

presently activated chord If a nonchord tone appears the listener will searchfor a solution that allows the incorporation of the nonchord tone in thepresent harmonic frame notably by establishing whether the nonchord toneis resolved by anchoring to a subsequent chord tone or incorporated in arun Thus it is essential that the nonchord tone is somehow incorporated inthe harmonic frame If this does not happen the formation of a valid har-monic interpretation fails and the sequence will receive a low rating Sincethe number of possible continuations tends to be highly constrained somenonchord tones (especially chromatic tones) may appear to be completelyincompatible with the present harmonic analysis and therefore be heard asa wrong tone

In a comparison of our explanation with a few alternative ones it wasfound that only the explanation based on Krumhanslrsquos tonal hierarchy con-cept could partly account for the data collected in Experiment 2 In theDiscussion of that experiment we have argued that this is probably due tothe fact that the average stability of the tones in a sequence is related to theharmonic factors operative in our explanation

Specific features in the stimuli such as the number of contour changesthe average consonance of the intervals and the variation of the intervalsizes appeared not to contribute to explaining the data

The finding that the data cannot be explained by the two principles thataccording to Schellenberg (1997) represent an aspect of the implication-realization theory of Narmour (1990 1992) may in first instance be sur-prising given the considerable body of research showing the importance ofthese principles in predicting expectancies listeners create when hearing animplicative interval However this result should be evaluated in the light ofthe following considerations In the studies testing the innate aspects ofNarmourrsquos model the principles were tested and shown to be operative inpredicting the expectations concerning the last tone of three-tone sequences(Cuddy amp Lunney 1995) or of the last tone of initial fragments of actualmelodies (Krumhansl 1995 Schellenberg 1996 1997) As mentioned ear-lier in the latter studies the influence of metrical and harmonic factors washeld constant as much as possible so that the role of these factors wasminimized In contrast in the present research the main variation concernsthe position and context of nonchord tones whereas factors related to thecontour were not systematically varied Moreover in the studies testingNarmourrsquos principles listeners focused on the final three tones when ratinghow well the last tone continued the melodic fragment whereas in ourexperiments which tested a model of melody processing listeners ratedthe goodness of entire melodies For these reasons it is understandablethat we did not find significant effects of Narmourrsquos principles in the presentresearch Furthermore it should be noted that the expectations based onNarmourrsquos principles pertain to contour constraints that are less restrictive(eg listeners have a general expectation for a small interval or after hav-

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 33: 2002Povel&Jansen

83Harmonic Factors in Melody Perception

ing heard a large ascending interval expect a subsequent smaller descend-ing interval) than those based on our model in which listenersrsquo expecta-tions are supposed to depend on the previous chord andor the occurrenceof a nonchord tone implying a specific resolution We want to emphasizethat this test of Schellenbergrsquos principles only bears on a relatively smallaspect of Narmourrsquos comprehensive implication-realization theory and thatthe results have no implication for this theory as a whole

In conclusion then our results seem to indicate that in the processing oftonal melodies meter and harmony are the primary frames in which a tonalmelody is described and that the precise contour of the melody may onlybe coded in some later stage if required Usually listeners can tell whether amelody is a common tonal melody before they can reproduce it But even ifparticipants are explicitly asked to reproduce a melody exactly it appearsthat they have trouble doing that and instead provide a paraphrase com-patible with its metrical and harmonic frame (Sloboda amp Parker 1985)

Although the results reported in this study give an impression of themechanisms that play a role in the induction of harmony additional re-search is needed to explicate the conditions in which anchoring and incor-poration in a run are effective in assimilating nonchord tones in the genera-tion of a harmonic analysis Using the insights gained so far we are presentlydeveloping a computational model that describes how a harmonic frame isdeveloped while a sequence evolves which expectations are generated atdifferent points in the sequence and which tones will lead to a breakdownof the harmonic analysis2

References

Bharucha J J (1984) Anchoring effects in music The resolution of dissonance CognitivePsychology 16 485ndash518

Bharucha J J (1987) Music cognition and perceptual facilitation A connectionist frame-work Music Perception 5 1ndash30

Bharucha J J (1991) Pitch harmony and neural nets In P M Todd amp D G Loy (Eds)Music and connectionism Cambridge MA MIT Press

Bharucha J J (1996) Melodic anchoring Music Perception 13 383ndash401Cuddy L L Cohen A amp Mewhort D J (1981) Perception of structure in short melodic

sequences Journal of Experimental Psychology Human Perception and Performance 7869ndash883

2 We thank Ric Ashley for his advice in constructing the stimuli and for allowing us torun the experiment in his music cognition classes at Northwestern University GlennSchellenberg for his advice and help in defining the factors related to Narmourrsquos model andEugene Narmour for several thoughtful remarks pertaining to the interpretation of theresults We gratefully acknowledge the support and encouragement of Bob Gjerdingen inconducting the comparison studies We thank Jules Ellis for his advice on the statisticalanalyses Hubert Voogd for helping develop the program that ran the experiments andReneacute van Egmond and Hank Heijink for a critical reading of an earlier version of thisreport

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 34: 2002Povel&Jansen

84 Dirk-Jan Povel amp Erik Jansen

Cuddy L L amp Lunney C A (1995) Expectancies generated by melodic intervals Per-ceptual judgments of melodic continuity Perception amp Psychophysics 57 451ndash462

Essens P amp Povel D J (1985) Metrical and non-metrical representations of temporalpatterns Perception amp Psychophysics 37 1ndash7

Gjerdingen R O (1990) Categorization of musical patterns by self-organizing neuron-like networks Music Perception 8 339ndash370

Holleran S Jones M R amp Butler D (1995) Perceiving implied harmony The influenceof melodic and harmonic context Journal of Experimental Psychology LMC 21 737ndash753

Jansen E L amp Povel D J (1999) Mechanisms in the perception of accented melodicsequences Proceedings of the 1999 Conference of the Society for Music Perception andCognition (p 29) Evanston IL Society for Music Perception and Cognition

Jansen E L amp Povel D J (2000) The role of implied harmony in the perception of brieftone sequences In C Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds)Proceedings of the Sixth International Conference on Music Perception and Cognition[CD-ROM] (pp 927ndash934) Keele Staffordshire UK Department of Psychology

Krumhansl C L (1990) Cognitive foundations of musical pitch New York Oxford Uni-versity Press

Krumhansl C L (1995) Music psychology and music theory Problems and prospectsMusic Theory Spectrum 17 53ndash80

Krumhansl C L amp Kessler E J (1982) Tracing the dynamic changes in perceived tonalorganization in a spatial representation of musical keys Psychological Review 89 334ndash368

Lerdahl F (1988) Tonal pitch space Music Perception 5 315ndash350Lerdahl F amp Jackendoff R (1983) A generative theory of tonal music Cambridge MA

MIT PressLonguet-Higgins H C amp Steedman M J (1971) On interpreting Bach In B Meltzer amp

D Michie (Eds) Machine intelligence Edinburgh University PressMalmberg C F (1918) The perception of consonance and dissonance Psychological Mono-

graphs 25 (2 whole No 108) 93ndash133Narmour E (1990) The analysis and cognition of basic melodic structures The implica-

tion-realization model Chicago University of Chicago PressNarmour E (1992) The analysis and cognition of melodic complexity The implication-

realization model Chicago University of Chicago PressPiston W amp Devoto M (1989) Harmony London Victor Gollancz (Original work

published 1941)Platt J R amp Racine R J (1994) Detection of implied harmony changes in triadic melo-

dies Music Perception 11 243ndash264Povel D J (1981) Internal representation of simple temporal patterns Journal of Experi-

mental Psychology Human Perception and Performance 7 3ndash18Povel D J amp Essens P (1985) Perception of temporal patterns Music Perception 2

411ndash440Povel D J amp Egmond R van (1993) The function of accompanying chords in the recog-

nition of musical fragments Music Perception 11 101ndash115Povel D J amp Jansen E (2000) Towards an on-line model of music perception In C

Woods G Luck R Brochard F Seddon amp J A Sloboda (Eds) Proceedings of theSixth International Conference on Music Perception and Cognition Keele UK KeeleUniversity Department of Psychology

Povel D J amp Jansen E (2001) Perceptual mechanisms in music perception Music Per-ception 19 169ndash198

Schellenberg E G (1996) Expectancy in melody Tests of the implication-realization modelCognition 58 75ndash125

Schellenberg E G (1997) Simplifying the implication-realization model of melodic ex-pectancy Music Perception 14 295ndash318

Sloboda J A amp Parker D H H (1985) Immediate recall of melodies In P Howell ICross amp R West (Eds) Musical structure and cognition London Academic Press

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50

Page 35: 2002Povel&Jansen

85Harmonic Factors in Melody Perception

Tabachnik B G amp Fidell L S (2001) Using multivariate statistics Boston Allyn ampBacon

Temperley D (1997) An algorithm for harmonic analysis Music Perception 15 31ndash68Thompson W F (1993) Modeling perceived relationships between melody harmony and

key Perception amp Psychophysics 53 13ndash24Thompson W F amp Cuddy L L (1989) Sensitivity to key change in chorale sequences A

comparison of single voices and four-voice harmony Music Perception 7 151ndash168Thompson W F amp Stainton M (1998) Expectancy in Bohemian folk song melodies

Evaluation of implicative principles for implicative and closural intervals Music Percep-tion 15 231ndash252

Tillmann B Bharucha J J amp Bigand E (2000) Implicit learning of tonality A self-organizing approach Psychological Review 107 885ndash913

Trainor L J amp Trehub S E (1992) A comparison of infants and adults sensitivity toWestern musical structure Journal of Experimental Psychology Human Perception ampPerformance 18 394ndash402

Trehub S E Thorpe L A amp Trainor L J (1990) Infantrsquos perception of good and badmelodies Psychomusicology 9 5ndash19

Van Dyke Bingham W (1910) Studies in melody Psychological Review Monograph Supple-ments Vol XII Whole No 50