Woodcock Johnson III Tests of Achievement Transition Academy @ Shadd October 5, 2010 (Adapted from...

Woodcock Johnson III

Tests of Achievement

Transition Academy @ Shadd

October 5, 2010

(Adapted from Riverside Publishing Materials)

Purpose of Academic Testing

• Identify areas of academic strength and weakness

• Diagnose specific learning disabilities• Develop recommendations to address

academic deficiencies

Learning Disabilities

• Two operational definitions:– Significant discrepancy between ability and

achievement– Response to intervention

• Different from DSM-IV-TR Learning Disorder, yet related.

Issues with Academic Testing

• Age norms vs. Grade norms– Kids enter school at different ages– Children can be accelerated or retained in grade– Which normative sample to use?

• If you are computing ability-achievement discrepancies, you will need to use age norms.

• However, if the student has been retained or accelerated in grade, should also re-compute scores using grade norms.

Psychometrics of WJ-III ACH

• Major strength of the WJ-III ACH test battery – co-normed with the WJ-III COG test battery.

• Standardization sample: 8,818 subjects• 100+ geographic areas (including Moorpark!)• Reliability:

– For individual subtests across all ages: .70 to .96

– For clusters across all ages: .93 to .99

– Alternative form reliabilities for Passage Comprehension range between .85 and .96 over all ages.

Psychometrics of WJ-III ACH

• Validity– Differential correlational patterns between different

cluster scores suggest discriminant validity.– Factor structure aligns with the narrow Cattell-Horn-

Carroll abilities• Validity studies with the KTEA and WIAT show

modest agreement– Total Achievement on WJ-III with KTEA: .79– Total Achievement on WJ-III with WIAT: .65– Convergent and Discriminant validity coefficients are

a little higher e.g.:• WJ-III Broad Reading with WIAT Reading Composite: .82• WJ-III Broad Reading with WIAT Math Composite: .50

Psychometrics of WJ-III ACH

• Discriminant validity with WJ-III COG, WAIS-III, and Kaufman Adolescent & Adult Intelligence testing produces lower coefficients (most below .50).

Organization

• Achievement (BLUE)– Standard Battery– 12 Tests– 10 Clusters

– Extended Battery– 10 Tests– 9 Clusters

– (Form A & B)

Clusters

• Broad Reading• Basic Reading Skills• Broad Math• Math Calculation Skills• Math Reasoning• Broad Written Language• Written Expression• Academic Knowledge• Academic Skills• Academic Fluency• Oral Language

Cluster and Subtest Structure

• Broad Reading– Letter-Word Identification– Reading Fluency– Passage Comprehension

• Basic Reading Skills– Letter-Word Identification– Word Attack (extended battery, commonly

administered)• Broad Math

– Calculation– Math Fluency– Applied Problems

Cluster and Subtest Structure

• Math Calculation Skills– Calculation– Math Fluency

• Math Reasoning– Applied Problems– Quantitative Concepts (extended battery)

• Broad Written Language– Spelling– Writing Fluency– Writing Samples

Cluster and Subtest Structure

• Written Expression– Writing Fluency

– Writing Samples

• Academic Knowledge– Academic knowledge (3 extended subtests: Sci, SS,

Hum)

• Academic Skills– Letter-Word Identification

– Spelling

– Calculation

Cluster and Subtest Structure

• Academic Fluency– Reading Fluency– Writing Fluency– Math Fluency

• Oral Language– Story Recall– Understanding Directions

Cluster and Subtest Structure

• Total Achievement– Letter-Word Identification

– Reading Fluency

– Passage Comprehension

– Spelling

– Writing Fluency

– Writing Samples

– Calculation

– Math Fluency

– Applied Problems

“IQ tests are a poor way to judge people's abilities, failing as they do to account for magic, which has its own importance, both by

itself and as a compliment to logic.”

- Robert James Waller

Test Administration: Tools & Techniques

• Strive for exact & brisk administration• Have necessary materials

– Test books– Test records– Subject response booklet– Tapes– Tape player– Headphones, Y-jack, Earpiece– Pencils– Stopwatch/Timer

Administration Tool:Test Session Observation Checklist

• Level of conversational proficiency

• Level of cooperation• Level of activity• Attention & concentration• Self-confidence• Care in responding• Response to difficult tasks

Administration Time (ACH)

• Preliminary Testing at BC– Intake Booklet & – Ach Tests 2, 5, 6, 7, 8, 11– Approx. 120 minutes

• Standard Achievement– Tests 1, 3, 4, 9, 10– 60 minutes

• Extended Achievement – Tests 13 - 18– 60 minutes

Standard Battery

• Letter-Word Identification

• Reading Fluency (timed)

• Story Recall (taped)

• Understanding Directions (taped)

• Calculation

• Math Fluency (timed)

• Spelling

• Writing Fluency (timed)

• Passage Comprehension

• Applied Problems

• Writing Samples

• Story Recall – Delayed

Extended Battery

• Word Attack• Picture Vocabulary• Oral Comprehension

(taped)• Editing• Reading Vocabulary

• Quantitative Concepts• Academic Knowledge• Spelling of Sounds• Sound Awareness• Punctuation &

Capitalization

Administration Techniques• Know your tools

• Follow standardized procedures

• Use suggested starting points

• Know basal and ceiling rules

• Know pronunciations

• Don’t penalize mispronunciations due to speech deficits or regional differences

• Student must smoothly pronounce words for credit (automaticity is critical)

“If you think you are over your head with a client, you

probably are.”

- Richard Bolles

Administration Techniques• Test by complete pages when

subject sees items• Score LAST response given• Give benefit of doubt every other

time• Don’t repeat items unless

directed• Use pause button if needed• If interrupted by a sound, repeat

item later

Administration Techniques• Adhere to time limits• Record exact finishing times• On timed written tests, direct

student to cross out errors• Check questionable responses

later; proceed as if correct

“We never do anything well till we cease to think about the

manner of doing it.”

- William Hazlitt

“The IQ and the life expectancy of the average American

recently passed each other going in the opposite direction.”

- George Carlin

Tests of Achievement

Standard Battery

&

Extended Battery

Tests of Achievement

• Standard Battery– Letter-Word Identification– Reading Fluency– Story Recall– Understanding Directions– Calculation– Math Fluency– Spelling– Writing Fluency – Passage Comprehension– Applied Problems – Writing Samples– Story Recall – Delayed

• Extended Battery– Word Attack– Picture Vocabulary– Oral Comprehension– Editing– Reading Vocabulary– Quantitative Concepts– Academic Knowledge– Spelling of Sounds– Sound Awareness– Punctuation &

Capitalization

Standard Battery

• Test 1: Letter-Word Identification– Identifying printed letters and words

• Test 2: Reading Fluency– Reading printed statements rapidly and responding true or false

(Y or N)

• Test 3: Story Recall– Listening to and recalling details of stories

• Test 4: Understanding Directions– Listening to a sequence of instructions and then following the

directions (by pointing)

• Test 5: Calculation– Performing various mathematical calculations; retrieving math

facts.

Standard Battery

• Test 6: Math Fluency– Adding, subtracting, and multiplying rapidly

• Test 7: Spelling– Spelling orally presented words

• Test 8: Writing Fluency– Formulating and writing simple sentences rapidly with a visual

cue

• Test 9: Passage Comprehension– Completing a sentence by giving the missing key word that

makes sense in the context.

• Test 10: Applied Problems– Performing math calculations in response to orally presented

problems with a visual cue.

Standard Battery

• Test 11: Writing Samples– Writing meaningful sentences for a given purpose

with a visual and oral prompt.

• Test 12: Story Recall – Delayed– Recalling previously presented story elements

• Test 13: Word Attack– EXTENDED Battery test, often included in most

standard battery administrations– Reading phonically regular non-words

Extended Battery

• Test 13: Word Attack– Reading phonically regular non-words

• Test 14: Picture Vocabulary– Naming pictures

• Test 15: Oral Comprehension– Completing an oral sentence by giving the

missing key word that makes sense in the context• Test 16: Editing

– Identifying and correcting errors in written passages

Extended Battery

• Test 17: Reading Vocabulary– Reading words and supplying synonyms and

antonyms; reading and completing verbal analogies• Test 18: Quantitative Concepts

– Identifying math terms and formulae; identifying number patterns

• Test 19: Academic Knowledge– Responding to questions about science, social studies,

and humanities• Test 20: Spelling of Sounds

– Spelling letter combinations that are regular patterns in written English

Extended Battery

• Test 21: Sound Awareness– Providing rhyming words; deleting,

substituting, and reversing parts of words to make new words.

• Test 22: Punctuation & Capitalization– Applying punctuation and capitalization rules

“Sometimes when I'm in a bookstore or a library, I am

overwhelmed by all the things that I do not know.”

- Arthur C. Clarke

Basal and Ceiling Rules

• For almost all tests (exceptions are noted in the scoring record sheets), to obtain a valid score on any subtest, you need a basal level of 6 items correct and a ceiling level of 6 items incorrect.

• Add up the total number of items correct (including items BELOW basal) = Raw Score.

• Enter Raw Scores in the WJ-III Scoring Assistant (hand scoring not really an option).

Scoring Output

• W-scores– Equal-interval scale where 500 is set to approximate the average

performance of a 10-0 child. – Used to measure growth.

• Age Equivalents / Grade Equivalents– Raw score cross-referenced to the median score within the age/grade

band.– Highly variable.

• RPI – Relative Proficiency Index– Predicts a student’s level of proficiency on tasks that typical age- or

grade-level peers would perform with 90% proficiency.– RPI of 55/90 indicates impairment – they did 55%, a peer would do

90%– RPI of 98/90 indicates strength – they did 98%, a peer would do

90%

Scoring Output

• Percentile Ranks

• Standard Scores– Mean 100; SD 15– Used in determining ability-achievement

discrepancies.

• Z Scores

“Your mind should be a place where you work things out, not

store a bunch of stuff.”

- Albert Einstein

2. Reading Fluency

• Rapidly read simple sentences and mark whether it is true or false

• Start with Item 1• 3 minute time limit• Skipped items are scored “0”• If fewer than 3 correct on Practice Exercises C-F,

discontinue and score “0”• Enter Number Correct and Number Incorrect• Used in Broad Reading, Academic Fluency, Total

Achievement

3. Story Recall

• Use tape• Student listens to story on tape, then retells it• Must repeat Bold elements exactly; other parts may

be paraphrased (in any order)• Use suggested starting points• Follow continuation rules• Pause tape player after each story• Do not repeat or replay any stories• Used in Oral Language (Std & Ext), Oral Expression

4. Understanding Directions • Use Tape; pause tape if needed• Student points to items on picture according to taped

instructions• Allow student 10 seconds to view each picture before starting

tape• Use suggested starting points; follow discontinuation

instructions• Lay Test Manual flat• Tip: Only watch for correct answers – don’t try to figure it

out• Used in Oral Language (Std & Ext), Listening

Comprehension

6. Math Fluency

• Timed Math facts• Begin with Item 1• 3 minutes timed• Do not point out signs• Lay booklet open• Discontinue if 3 or fewer correct in 1 minute

• Used in Broad Math, Math Calculation Skills, Academic Fluency, Total Achievement

7. Spelling ()

• Spelling (not usage & punctuation)

• Basal/Ceiling = 6/6

• Used in Broad Written Language, Basic Writing Skills, Academic Skills, Total Achievement

8. Writing Fluency

• Student quickly writes simple sentences using prompt words

• Begin with Item 1• 7 Minutes timed• Discontinue if “0” correct on Samples or fewer

than 3 correct in first 2 minutes• Read stimulus words as needed• Stimulus words cannot be altered• Used in Broad Written Language, Written Expression,

Academic Fluency, Total Achievement

Tests of Achievement: Extended Battery

Word Attack Picture Vocabulary

Oral Comprehension Editing

Reading Vocabulary Quantitative Concepts

Academic Knowledge Spelling of Sounds

Sound Awareness Punctuation/Capitalization

Preliminary Testing (at BC)

Special Instructions Item Administered By:

Date & Initials

Intake Booklet & Consent Form

3 Minutes Timed Test 2: Reading Fluency

Test 5: Calculation

3 Minutes Timed Test 6: Math Fluency

Use Tape Test 7: Spelling

7 Minutes Timed Test 8: Writing Fluency

Use Tape Test 11: Writing Samples

Other:

“To avoid criticism, do nothing, say nothing, be nothing.”

- Elbert Hubbard

Scoring and Reporting

Normative Update

Normative Updates (Nu)

• Check accuracy of all scores• Launch Normative Update

Program– Set up program once:– Click <Options>– Click <Program Options>– Under “Additional Score

Column” choose CALP– For “Confidence Band” choose

68%– For “Discrepancy Cutoff”

choose 1.30

Normative Update Continued

• For Verbal & Numeric Data:• Enter scores for Cognitive & Achievement, Standard &

Extended Batteries• Enter observations• Click <Report> <Summary & Score Report>• Choose Age• Select “Include GIA/BIA”• For “Ability/Ach” choose “GIA-Extended”, then also run “Oral

Language” or “GIA-Standard”• For “Intra-Ability” choose “Intra-Cog and/or Intra-Ach”• Send to “Print Preview”• Click <OK>

Normative Update Report

• To see next page in report, press PAGE DOWN key

• Click icon to Print the report

• Click icon to Close Preview (or press ESC)

Normative Update Continued

• For Graphical Data:• Click <Report>, <Age/Grade

Profiles>• Choose “Age”• Choose “Include GIA/BIA”• For “Data to Include” choose

“Standard Score”• Send report to “Print Preview” • Click <OK>

“Machines increase the number of things we can do without thinking. Things we do without thinking -

there's the real danger.”

- Frank Herbert