Reading Assessments for Elementary Schools

Tracey E. Hall

Center for Applied Special Technology

Marley W. WatkinsPennsylvania State University

Frank C. WorrellUniversity of California, Berkeley

REVIEW: Major Concepts

• Nomothetic and Idiographic• Samples• Norms• Standardized Administration• Reliability• Validity

Nomothethic

• Relating to the abstract, the universal, the general.

• Nomothetic assessment focuses on the group as a unit.

• Refers to finding principles that are applicable on a broad level.

• For example, boys report higher math self-concepts than girls; girls report more depressive symptoms than boys..

Idiographic

• Relating to the concrete, the individual, the unique

• Idiographic assessment focuses on the individual student

• What type of phonemic awareness skills does Joe possess?

Populations and Samples I

• A population consists of all the representatives of a particular domain that you are interested in

• The domain could be people, behavior, curriculum (e.g. reading, math, spelling, ...

Populations and Samples II

• A sample is a subgroup that you actually draw from the population of interest

• Ideally, you want your sample to represent your population– people polled or examined, test content, manifestations of behavior

Samples

• A random sample is one in which each member of the population had an equal and independent chance of being selected.

• Random samples are important because the idea is to have a sample that represents the population fairly; an unbiased sample.

• A sample can be used to represent the population.

• Sampling in which elements are drawn according to some known probability structure.

• Probability samples are typically used in conjunction with subgroups (e.g., ethnicity, socioeconomic status, gender).

Norms I

• Norms are examples of how the “average” individual performs.

• Many of the tests and rating scales that are used to compare children in the US are norm-referenced.– An individual child’s performance is compared to the norms established using a representative sample.

Norms II

• For the score on a normed instrument to be valid, the person being assessed must belong to the population for which the test was normed

• If we wish to apply the test to another group of people, we need to establish norms for the new group

Norms III

• To create new norms, we need to do a number of things:– Get a representative sample of new population

– Administer the instrument to the sample in a standardized fashion.

– Examine the reliability and validity of the instrument with that new sample

– Determine how we are going to report on scores and create the appropriate tables

Standardized Administration

• All measurement has error.• Standardized administration is one way to reduce error due to examiner/clinician effects.

• For example, consider these questions with different facial expressions and tone:

• Please define a noun for me :-)• DEFINE a noun if you can ? :- (

Normal Curve

• Many distributions of human traits form a normal curve

• Most cases cluster near middle, with fewer individuals at extremes; symmetrical

• We know how the population is distributed based on the normal curve

Ways of Reporting Scores

• Mean, standard deviation• Distribution of scores

– 68.26% ± 1; 95.44 ± 2; 99.72 ±3• Stanines (1, 2, 3, 4, 5, 6, 7, 8, 9)

• Standard scores - linear transformations of scores, but easier to interpret

• Percentile ranks* • Box and Whisker Plots*

Percentiles

• A way of reporting where a person falls on a distribution.

• The percentile rank of a score tells you how many people obtained a score equal to or lower than that score.

• Box and whisker plots are visual displays or graphic representations of the shape of a distribution using percentiles.

The box plot is a picture of the distribution of scores on a measure.

Explanation of the

Box Plot

Individual

Outliers

90th Percentile

Performance

75th Percentile

50th

Percentile

25th Percentile

10th Percentile

0

2

4

6

8

10

12

14

16

18

20

Grade 2 Students

Correlation

• We need to understand the correlation coefficient to understand the manual

• The correlation coefficient, r, quantifies the relationship between two sets of scores

• A correlation coefficient can have a range from -1 to + 1– Zero means the two sets of scores are not related.

– One means the two sets of scores are identical (a perfect correlation)

Correlation 2

• Correlations can be positive or negative.• A + correlation tells us that as one set of scores increases, the second set of scores also increases. they can be negative. Examples?

• A negative correlation tells us that as one set of scores increases, the other set decreases. Think of some examples of variables with negative r’s.

• The absolute value of a correlation indicates the strength of the relationship. Thus .55 is equal in strength to -.55.

How would you describe the correlations shown

by these charts?3 15 47 39 7

11 913 10

1098765

1.21.21.21.21.2

0

2

4

6

8

10

12

14

1 2 3 4 5 6

Series1

0

2

4

6

8

10

12

1 2 3 4 5 6

Series1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1 2 3 4 5

Series1

Reliability

• Reliability addresses the stability, consistency, or reproducibility of scores.– Internal consistency

– Split half, Cronbach’s alpha

– Test-retest– Parallel/Alternate forms– Inter-rater

Validity

• Validity addresses the accuracy or truthfulness of scores. Are they measuring what we want them to?– Content– Criterion - Concurrent– Criterion - Predictive– Construct

– Face– (Cash)

Content Validity

• Is the assessment tool representative of the domain (behavior, curriculum) being measured?

• An assessment tool is scrutinized for its (a) completeness or representativeness, (b) appropriateness, (c) format, and (d) bias– E.g., MSPAS

Criterion-related Validity

• What is the correlation between our instrument, scale, or test and another variable that measures the same thing, or measures something that is very close to ours?

• In concurrent validity, we compare scores on the instrument we are validating to scores on another variable that are obtained at the same time.

• In predictive validity, we compare scores on the instrument we are validating to scores on another variable that are obtained at some future time.

Construct Validity

• Overarching construct: Is the instrument measuring what it is supposed to?– Dependent on reliability, content and criterion-related validity.

• We also look at some other types of validity some times– Convergent validity: r with similar construct

– Discriminant validity: r with unrelated construct

– Structural validity: What is the structure of the scores on this instrument?

Elementary Normative Sample

• Stratified by educational region

• Males and females represented equally.

• School, class, and individuals chosen at random.

• Final sample consists of 700 students (50% female).

Table 1.1

Normative Sample by Educational Region

Population Norm Sample

Division Number Percent Number PercentSt. George West 31,948 24.1 160 22.9St. George East 14,255 16.5 116 16.6St. Andrew/St. David 3,859 6.3 44 6.3Caroni 10,913 15.0 105 15.0Nariva/M ayaro 2,287 3.7 29 4.1Victoria 26,197 19.0 133 19.0St. Patrick 12,711 11.0 80 11.4Tobago 3,059 4.3 33 4.7Total 105,229 100.0 700 100.0

p. 2

Table 1.2

Normative Sample by Age

Age Number Percent

4 18 2.6 5 88 12.6 6 96 13.7 7 98 14.0 8 98 14.0 9 99 14.1 10 95 13.6 11 89 12.7 12 14 2.0 13 4 0.6 14 1 0.1 p. 2

Table 1.3

Normative Sample by Gender

Gender Number Percent

Male 350 50.0Female 350 50.0

p. 2

Table 1.4

Normative Sample by Grade

Grade Number Percent

1st Year 100 14.32nd Year 100 14.3Standard 1 100 14.3Standard 2 100 14.3Standard 3 100 14.3Standard 4 100 14.3Standard 5 100 14.3

p. 3

Table 1.5

Normative Sample by Ethnic Background

Ethnic Background Number Percent

African 269 38.4East Indian 257 36.7M ixed 149 21.3Other 1 0.1Not Reported 24 3.4

p. 3

Table 1.6

Normative Sample by Parental Education Level

Mothers Fathers

Highest level completed Number Percent Number Percent

Primary 209 29.9 216 30.9Form 3 49 7.0 31 4.4Form 5 203 29.0 169 24.1Form 6 15 2.1 16 2.3University 5 0.7 10 1.4Unknown 219 31.3 258 36.9

p. 3

Measures

• First and Second Year/Infants 1 and 2– Mountain Shadows Phonemic Awareness Scale (MS-PAS) - group administered.

– Individual Phonemic Analysis

• Second Year/Infant 2 to Standard 5– Oral Reading Fluency

• Standards 1 and 2– The Cloze Procedure

Assessment• Determine starting point • Analyze Errors• Monitor Progress• Modify Instruction

Instructional Delivery• Secure student attention• Pace instruction appropriately• Monitor student performance• Provide feedback

Instructional Design• Determine Content• Select Language of Instruction• Select examples• Schedule scope and sequence• Provide for cumulative review

Initial Evaluation• Archival Assessment• Diagnostic Assessments• Formal Standardized Measures

Madigan, Hall, & Glang(1997)

Assessment Instruction Cycle