Standardized test analysis project

Primary and Intermediate Measures of Music

Audiation

Carlee Francisco & Morgan Soja

Bibliographic Information

PMMA

• Edwin Gordon, GIA Publications Inc., 1979, 1986.

• $100 for the complete kit• http://www.giamusic.com

IMMA

• Edwin Gordon, GIA Publications Inc., 1979, 1986.

• $100 for the complete kit• http://www.giamusic.com• Other products available

Purpose of Tests

• Designed to act as an objective aid • Teachers and parents • Help each child make the best use of his or her

music aptitudes • Provide appropriate opportunities and instruction

PMMA and IMMA:

Purpose of Tests cont.

• Used periodically to evaluate tonal and rhythm aptitudes. Comparing scores helps inform instruction for child’s individual music needs.

• Used periodically to identify high achieving children who can profit from additional music activities. (PMMA is not designed to exclude).

• Used periodically to provide normative analysis (comparing each child with other children) to inform instruction and further the child’s individual music needs.

What is audiation?

• Gordon states: “Audiation takes places when one hears and feels music through recall or creativity, the sound not being physically present except when one is audiating while also aurally perceiving music that is being performed by others or that one is performing himself.”

• PMMA and IMMA = immediate reaction– No memory involved

• Individualized instruction requires understanding of aptitude

• Developmental music aptitude for young children seems directly concerned with same and different

• Stabilized music aptitude for older children and adults seems indirectly concerned with same and different

• Children and adults audiate differently thus, learn music differently, thus must be taught differently

Level

PMMA

• K-Grade 3• Teaching 4-12, not

necessarily appropriate• Test audiation, identify

students for remediation

IMMA

• Grades 1-6 • Developmental aptitude, AND

stabilized aptitude• Students with high

achievement (80%+) on PMMA

• Changes in aptitude cannot be made by comparing scores on PMMA and IMMA.

• 5-6th grade chorus

Validity

• Content• Concurrent

• Inverse Concurrent• Congruent

• Longitudinal Predictive

Content Validity

• Gordon states: “unless the teacher who uses a test is satisfied that the test content fairly represents factors which should be measured, he will have no confidence in the results of the tests, even if the objective validity of the test is overwhelming.”

• Review of the manual, the purpose, the studies that provide the basis for the tests.

Intercorrelation of Test Items

• Intercorrelation was significant at the 1% level if the coefficient was .23 or higher.

• Only pairs of items that were intercorrelated at ±.30 or higher were used on the tests.

Intercorrelation of Test Items cont.

• 26% (422) of tonal pattern intercorrelations ≤ .30 were positive, 8% negative (34% or 544 total)

• 17% (277) of rhythm pattern intercorrelations ≤ .30 were positive, 9% (147) negative (26% or 424 total)

• Lower rhythm is likely due to lower reliability on rhythm test items and test.

Concurrent Validity

• Gordon notes that correlation of test scores with teacher given grades is not the best measure of validity for a test of music aptitude. – Constant variation of developmental music aptitude

Concurrent Validity cont.

Chesnut Hill Academy – Philadelphia, PAPrivate boys school

Music teacher asked to rate children on 5-point scale for overall music aptitude.

Grade # First 24Second 23Third 29Fourth 31

Chesnut Hill Academy cont.

Music teacher ratings were correlated with IMMA scores

* Note low rhythm correlations – emphasis of need to use test to diagnose musical strengths and weaknesses in all children

Tonal Rhythm CompositeGrade 1 0.45 0.23 0.35Grade 2 0.47 0.22 0.44Grade 3 0.46 0.25 0.45Grade 4 0.4 0.25 0.36


Jenkintown, PA - Elementary School

Band director (4th Grade) asked to rate children on a five-point scale for overall music aptitude

32 students in fourth grade – 21 in instrumental class (group who received ratings)

IMMA was administered to children after ratings given

Jenkintown, PA cont.

Music teacher ratings were correlated with IMMA scores

Follow up study was conducted

Tonal 0.42Rhythm 0.69Composite 0.81

Jenkintown, PA cont.

11 students that were not involved in instrumental class

No real differences in the means of groups*Highest scoring student was in non-instrumental group

Instrumental Students Non-Instrumental StudentsMean SD Mean SD

Tonal 36.4 1.66 36.6 1.91Rhythm 32.7 3.09 32.8 3.34Composite 69.1 4.01 69.4 4.03


West Irondequoit – Kindergarten 1st, 2nd , 3rd

Means and standard deviations for PMMA given approximately two weeks apart show less than 1.3 points gained or lost when students were re-tested

Practice effects do not effect validity of the PMMA or IMMA

Concurrent Validity cont.PMMA Test-Retest Means and Standard Deviations

First Administration Second Administration Mean DifferencesMean SD Mean SD

TonalK 24.7 5.28 24.4 5.50 -0.3Grade One 29.8 5.03 29.2 5.04 -0.6Grade Two 32.0 4.75 31.8 4.54 -0.2Grade Three 34.6 3.35 33.6 4.13 -1.0

RhythmK 22.3 3.74 22.5 3.83 0.2Grade One 25.8 4.34 25.5 4.91 -0.3Grade Two 27.7 4.55 27.4 4.80 -0.3Grade Three 29.4 3.99 29.2 4.41 -0.2

CompositeK 47.0 7.65 46.9 8.52 -0.1Grade One 55.6 8.25 54.7 8.54 -0.9Grade Two 59.7 8.35 59.2 8.54 -0.5Grade Three 64.0 6.29 62.8 7.61 -1.2

Inverse Concurrent Validity

• PMMA and IMMA should have little in common with tests that are designed for purposes other than measuring music aptitude

• PMMA scores (tonal, rhythm, composite) were correlated with scores on academic achievement and intelligence tests.

Inverse Concurrent Validity cont.

Kindergarten

PMMA Metropolitan Readiness Tests Level TwoLanguage Total Quantitative Total Readiness Total

Tonal 0.29 0.24 0.30Rhythm 0.16 0.15 0.17Composite 0.29 0.23 0.30


Grade One

PMMA Stanford Achievement TestTotal Reading Total Math Complete Battery



Grade Two

PMMA Stanford Achievement TestTotal Auditory Total Reading Total Math



Grade Three

PMMA Stanford Achievement TestTotal Auditory Total Reading Total Math



Grade Three

PMMA Lorge Thorndike Intelligence TestsVerbal Nonverbal

Tonal 0.19 0.20Rhythm 0.25 0.29Composite 0.26 0.30


Correlations between PMMA and academic/intelligence tests are low

Correlations are similar to that of MAP and academic/intelligence tests

Valid music aptitude tests should have no more than 15% of their variance in common with academic tests.

What is in common is likely due to behaviors that are similar.

Congruent Validity

Correlation of two tests that are designed for the same purposes

Examination of PMMA, IMMA and MAP

Congruent Validity cont.

Great Valley, PAPMMA and IMMA administered approximately one month apart

Grade # First 26Second 26Third 34Fourth 40


Correlations of PMMA and IMMA

Tonal Rhythm CompositeGrade 1 0.58 0.53 0.62Grade 2 0.68 0.56 0.72Grade 3 0.62 0.64 0.66Grade 4 0.51 0.60 0.74


Correlations of PMMA and IMMA are almost as high as split halves reliabilities for IMMA

The two tests share much in common

High scores on PMMA suggest need to administer IMMA


Troy, NY – IMMA and MAP -Only Tonal Imagery and Rhythm Imagery sections of

MAP were used

MAP IM MATonal Rhythm

Tonal Imagery - Melody 0.54Tonal Imagery - Harmony 0.50Tonal Imagery - Total 0.58

Rhythm Imagery - Tempo 0.52Rhythm Imagery - Meter 0.60Rhythm Imagery - Total 0.63


IMMA has as much in common with the MAP as it has with the PMMA

Valid for developmental and stabilized music aptitude

Congruent Validity cont.West Irondequoit, NY – 4th Grade

PMMA and MAP administered to 227 children in 4th grade over 3 week period

MAP Primary Measures of Music AudiationTonal Rhythm Composite

Tonal Imagery - Melody 0.39 0.33 0.43*0.51* *0.45* *0.56*

Tonal Imagery - Harmony 0.29 0.24 0.31*0.40* *0.34* *0.42*

Tonal Imagery - Total 0.34 0.51 0.55*0.44* *0.70* *0.71*

Rhythm Imagery - Tempo 0.25 0.42 0.44*0.34* *0.60* *0.58*

Rhythm Imagery - Meter 0.35 0.53 0.56*0.43* *0.68* *0.68*

Rhythm Imagery - Total 0.35 0.52 0.61*0.43* *0.63* *0.71*

Composite 0.4 0.52 0.61*0.47* *0.63* *0.71*


High scores for 4th graders

Ceiling effect

Split halves reliabilities:

Coefficients were corrected

PMMA has more in common with IMMA than MAP

Tonal 0.80Rhythm 0.75Composite 0.82

Longitudinal Predictive Validity

Plymouth Meeting, PA – Parochial School

Students required to take violin for one year

Just after receiving violins and/or playing for no longer than a year, the PMMA was administered to 26 students who were 7 or 8 years old

At the end of one year of violin instruction, students performed on violin and were rated in three categories by two judges

Longitudinal Predictive Validity cont.

PMMA and Instrumental Performance Achievement

One year predictive validity coefficients for PMMA are comparable to three year predictive validity coefficients for MAP

PMMA Intonation Rhythm Expression CompositeTonal 0.75 0.50 0.71 0.78Rhythm 0.66 0.37 0.54 0.58Composite 0.74 0.42 0.68 0.73


Violin teacher rated students (on a five point scale) in overall music aptitude after teaching them for one year

Validity coefficients are lower than longitudinal predictive validity coefficients.

-Teacher’s knowledge of music aptitude is less accurate than what PMMA can determine even before instruction

PMMA Teacher RatingsTonal 0.33Rhythm 0.49Composite 0.46


Hunter College Elementary – NYC, NYFive year old children from two classes (n=13, and n=10)

PMMA administered before 12 week group violin lessons were taught by one instructor

Children performed by rote two musical phrases with rhythmic variation

Children’s performances were reviewed by two judges, whose ratings were correlated with PMMA scores


Hunter College Elementary – NYC, NYPMMA Longitudinal Predictive Validity Coefficients using

Instrumental Performance Achievement

Group 1 n = 13 Group 2 n = 10PMMA Tonal Rhythm Composite Tonal Rhythm CompositeTonal 0.67 0.31Rhythm 0.41 0.02Composite 0.59 0.15


Chesnut Hill, PA

33 students, aged nine ( 4th grade) studied violin and recorder for one semester each

IMMA was administered before instrumental instruction

After instructional period, students sang two songs that they then performed on their instrument

Two judges evaluated melodic and rhythmic accuracy and musical expression


Chesnut Hill, PAIMMA Longitudinal Predictive Validity Coefficients using Ratings of

Instrumental Performance Achievement

Recorder Group Violin GroupFirst Semester Instrument Singing Instrument SingingIMMATonal 0.74 0.63 0.54 0.57Rhythm 0.76 0.69 0.71 0.74Composite 0.75 0.67 0.63 0.67

Second SemesterIMMATonal 0.69 0.62 0.9 0.63Rhythm 0.79 0.74 0.91 0.76Composite 0.74 0.69 0.92 0.7


Chesnut Hill, PACoefficients are similar to predictive validity coefficients for the PMMA and the MAP

To study practice and training effects, on the IMMA tests were given at the end of the instructional period

Negligible differences

IMMA is a valid test of developmental and stabilized music aptitude, and practice and training have no effect on a valid test of stabilized music aptitude

Longitudinal Predictive Validity cont.IMMA Pre and Post Instruction Means and SD

IMMA Pre-and Post Instruction Correlations

Pre-Instruction Post-Instr uctionTonal Rhythm Composite Tonal Rhythm Composite

Recorder/ Violin

Mean 34.8 31.2 66.0 35.0 32.0 67.0SD 8.20 8.34 17.0 8.88 8.53 17.1

Violin/ Recorder

Mean 33.9 31.3 65.0 34.7 31.4 66.1SD 8.90 8.42 17.1 9.14 8.41 17.4

IMMA Recorder/Violin Violin/RecorderTonal 0.98 0.96Rhythm 0.95 0.97Composite 0.99 0.98

Reliability

• Split-Halves Reliability• Test-Retest Reliability

• Standard Error of Measurement

Reliability

PMMA Reliability and SETonal Rhythm Composite

Reliability Reliability ReliabilityKindergartenSplit-Halves 0.85 0.72 0.90Test-Restest 0.73 0.60 0.74Standard Error of Measure 1.8 2.0 2.3

Grade 1 .Split-Halves 0.89 0.85 0.92Test-Retest 0.70 0.66 0.75Standard Error of Measure 1.5 1.7 2.5

Grade 2Split-Halves 0.89 0.86 0.92Test-Retest 0.7 0.73 0.76Standard Error of Measure 1.4 1.8 2.5

Grade 3Split-Halves 0.85 0.86 0.90Test-Retest 0.68 0.66 0.73Standard Error of Measure 1.3 1.6 1.9

Reliability

IMMA Reliability and SETonal Rhythm Composite

Reliability Reliability ReliabilityGrade 1Split-Halves 0.76 0.70 0.80Test-Retest Raw 0.88 0.84 0.91Test-Retest Criterion 0.78 0.76 0.81Standard Error of Measure 1.8 1.7 2.4Grade 2 .Split-Halves 0.78 0.72 0.82Test-Retest Raw 0.87 0.82 0.91Test-Retest Criterion 0.79 0.75 0.80Standard Error of Measure 1.6 1.6 2.3Grade 3Split-Halves 0.74 0.71 0.81Test-Retest Raw 0.87 0.81 0.9Test-Retest Criterion 0.81 0.72 0.78Standard Error of Measure 1.3 1.5 2.0Grade 4Split-Halves 0.72 0.7 0.80Test-Retest Raw 0.85 0.83 0.90Test-Retest Criterion 0.86 0.84 0.76Standard Error of Measure 1.1 1.3 1.5

Reliability

Tonal test reliabilities were higher

Could be due to difficulty with audiation of rhythm patterns for young children

IMMA includes criterion scores used for indentifying students with exceptionally high music aptitude

Children who achieved the criterion score or higher received a 2, and those who did not were awarded a 1.

Test-Retest reliabilities for criterion scores are higher than the split-halves reliabilities and lower than the test-retest reliabilities of the raw scores

Odd/Even scores were not used for IMMA

Abnormal distribution of item difficulties

Test Construction2 subtests – Tonal, Rhythm

Items presented on CD (although directions refer to cassette tape)

First example is announced then played, second example is announced, then played

Children circle two happy faces for “same”, one happy, one sad face for “different”

Directions are spoken (female voice), examples are played on synthesizer

Standardization Process

Instructions are clear

Audio device where children can hear

Answer sheets on desk when students come in, or given afterwards

K – students may sit on floor and spread out

Extra pencils for students

Student’s name on answer sheet (clearly)

Become familiar with recording before administering

Make sure children understand

Teacher may start and stop recording at any time

Clearly written directions in manual

Manual

Face validity

Attractiveness

Orderly and understandable

Necessary information provided

Usefulness to test administrator

Manual

Face validity

Attractiveness

Orderly and understandable

Necessary information provided

Usefulness to test administrator

Audio Recording

Quality of directions on recording is good

Quality of tonal examples is good

Rhythmic examples are difficult to hear at times

Presentation is currently in CD form

(previously cassette tape)

Scoring & Answer Sheet

Scoring masks provided = Easy, quick scoring

Accurate scoring with no machine

CONVERSION OF SCORES?

Answer sheet is clear and easily understandable

Use of pictures instead of numbers

Norms

Norms cont.

Item Discrimination

Item Difficulty

Uses

General Opinion

Documents

Standardized test analysis project