Revision, evolution and revolution in assessment: The … · Revision, evolution and revolution in assessment: ... A. Intro & Overview B. ... Fluent reading requires attention to

5/4/2015

1

Revision, evolution and revolution in assessment: The WJ IV and

beyond CHC theory

Dr. Kevin S. McGrew, PhD

Institute for Applied Psychometrics (IAP)

WJ IV Presentation Menu Dr. Kevin McGrew

A. Intro & Overview

B. New tests

C. Cognitive Complexity Principle (Brief)

D. WJ IV “Intelligent” testing (BB/TG)

E. COG Battery

F. The New Ga

G. OL Battery (+tech info)

H. ACH Battery (+tech info)

I. Variations & Comparisons

J. Advanced: Ref.-focused testing

N. IQs Corner fun stuff

E1. Scholastic Aptitudes

E2. COG tech

Pre The ?

B2. COG/OLB1. ACH

M. ASB’s

E3. WJ IV COG/Wechslers

L. ECAD

K. General technical info

O. Case study-Patrick

http://www.iapsych.com/IAPWEB/iapweb.html


A WJ IV intro and organizational overview 042714.pptx

A WJ IV intro and organizational overview 042714.pptx

D WJ IV CHC breaking bad 050315.pptx

D WJ IV CHC breaking bad 050315.pptx

F Ga no longer rodney dangerfield KMcGrew 050315.pptx

F Ga no longer rodney dangerfield KMcGrew 050315.pptx

G WJ IV ACH 091414.pptx

G WJ IV ACH 091414.pptx

N IQs Corner Fun stuff 050315.pptx

N IQs Corner Fun stuff 050315.pptx

B WJ IV new tests all 091414.pptx

B WJ IV new tests all 091414.pptx

intro and conclusions 061214.pptx

intro and conclusions 061214.pptx

intro 092414.pptx

intro 092414.pptx

conclusions 092414.pptx

conclusions 092414.pptx

C cognitive complexity brief 050415.pptx

C cognitive complexity brief 050415.pptx

E WJ IV COG overview 050314.pptx

E WJ IV COG overview 050314.pptx

O wjiv case study Patrick 050415.pptx

O wjiv case study Patrick 050415.pptx

E1 WJ IV SAPT overview KMcGrew 050315.pptx

E1 WJ IV SAPT overview KMcGrew 050315.pptx

E2 WJ IV COG tech info 050315.pptx

E2 WJ IV COG tech info 050315.pptx

preshow 092414.pptx

preshow 092414.pptx

wj4revolution.pptx

wj4revolution.pptx

B1 WJ IV new tests ACH 050415.pptx

B1 WJ IV new tests ACH 050415.pptx

B2 WJ IV new tests COGOL 050415.pptx

B2 WJ IV new tests COGOL 050415.pptx

E3 WJ IV COG and Wechslers.pptx

E3 WJ IV COG and Wechslers.pptx

M WJ IV ASBs.pptx

M WJ IV ASBs.pptx

H WJ IV ACH 050415.pptx

H WJ IV ACH 050415.pptx

J WJ IV ach periodic tables 050315.pptx

J WJ IV ach periodic tables 050315.pptx

L ECAD overview 050415.pptx

L ECAD overview 050415.pptx

K WJ IV general tech overview 050315.pptx

K WJ IV general tech overview 050315.pptx

I WJ IV variations comparisions 050315.pptx

I WJ IV variations comparisions 050315.pptx

5/4/2015

2

Use the most contemporary measurement model of the evolving CHC

model of human cognitive abilities

“Beyond CHC” – CHC plus contemporary neurocognitive

research

© Institute for Applied Psychometrics; Kevin S. McGrew 05-01-15

Evolution of CHC Theory in the WJ IV

WJ (1977): Scientific-Empirical (pragmatic)

WJ-R (1989): Extended Cattell-Horn Gf-Gc Theory

WJ III (2001): CHC Theory

WJ IV (2014): Beyond CHC Theory


5/4/2015

3

The CHC taxonomy of cognitive abilities codebook (v2.5)

GfGc Gwm

Glr

Gv

Ga

Gs

GqGrw

g

Download at: http://www.iapsych.com/chccodebook1.pdf

http://www.iapsych.com/chccodebook2.pdf

Or visit:

© Institute for Applied Psychometrics; Kevin McGrew 06-18-14

Organize the battery into three (four) independent, co-normed batteries that can

be used independently or in any combination to provide greater flexibility

for examiners


chccodebook1.pdf

C:/Users/Kevin/Documents/aaaPresent and Education/CHC new/chc101codebook061414.pdf

C:/Users/Kevin/Documents/aaaPresent and Education/CHC new/chc101codebook061414.pdf

chccodebook2.pdf

http://www.iqscorner.com/

http://www.iqscorner.com/

5/4/2015

4

Organization

WJ IV

Cognitive Achievement Oral Language

Standard Extended Standard Extended StandardBattery Battery Battery Battery Battery

ABC

10 tests 8 tests 11 tests 9 tests 12 tests 7 clusters 10 clusters 15 clusters 7 clusters 9 clusters

WJ IV COG and OL tests by CHC factor domains

Oral Vocabulary

(VL)

Number Series

(RQ)

General Information

(K0)

Object-Num.

Sequencing

(WM)

Pair Cancellation

(P/WM-AC)

Memory for Words

(MS)

Analysis-Synthesis

(RG)

Concept Formation

(I)

Numbers Reversed

(WM)

Picture Recognition

(MV)

Visual-Aud.

Learning

(MA)

Story Recall

(MM)

Number-Pat.

Matching

(P)

COG

OL

New or sign. change

Verbal Attention

(WM/AC)

Letter-Pat. Matching

(P)

Nonword Repetition

(PC/UM-MS)

Phonological Processing

(PC/Glr-LA)

Visualization

(Vz)

Picture Vocabulary

(VL/LD)

Oral Comp.

(LS)

Under.

Directions

(WM/Gc-LS)

Sentence Repetition

(MS)

Retrieval Fluency

(FI/LA)

Rapid Picture Naming

(NA/LA)

Sound Awareness

(PC)

Sound Blending

(PC)

Segmentation

(PC)

Comp -Knowledge

(Gc)

Fluid Reasoning (Gf)

Long-Term Retrieval (Glr)

Visual Processing (Gv)

Auditory Processing

(Ga)

Processing

Speed (Gs)

Short-Term

Wrk Mem(Gwm)


5/4/2015

5

Gc Gwm Gs Ga Glr GvGf

Clusters available from Standard Cognitive easel

(10 tests)

(#) = # tests

Cognitive Efficiency (2)

Other clusters available by combining Standard Cognitive

tests with select tests from Extended Cognitive easel (8 tests)


Aud. Proc. (2)

LT Ret.(2)

VisualProc. (2)

Cog. Proc.Speed(2)

Fld. Reas.(2)

Cmp. Knw.(2)

ST Work. Mem. (2)

General Intellectual Ability – g (7)

Brief Intellectual Ability (3)

Gf-GcComposite (4)

Scholastic Aptitude Clusters (each a mix of 4 CHC abilities)

Number Facility (2)

Perc.Speed(2)

Qnt. Reas.(3)

New

Aud. Proc. (2)

LT Ret.(2)

VisualProc. (2)

Cog. Proc.Speed(2)

Fld. Reas.(2)

Cmp. Knw.(2)

ST Work. Mem. (2)

ST Work. Mem. (3)

Cmp. Knw.(3)

Fld. Reas.(3)

© Institute for Applied Psychometrics; Kevin S.

McGrew 05-01-15

Clusters available from Oral Language easel (12 tests)

(#) = # tests


Oral Lang. (2) *

Broad Oral Language (3) *

Oral Expression (2)

New

(* English or Spanish)

PhoneticCoding (2)

Spd. Lexical Access (2)

Listening Comprehension (2) *

Provides an overall index of oral language ability in Spanish as well as in English

• New Spanish/English Comparative Language Index (CLI) scores *

English & Spanish Oral Language/Ach discrepancy comparisons


5/4/2015

6


Vocab.(2)

(These two clusters require one test each from Cognitive and Oral Language easels)

Aud. Mm. Sp.(2)

Organization of WJ IV Cognitive (COG) and Oral Language (OL) batteries

General Intellectual Ability – g (7)

Brief Intellectual Ability (3)

Gf-Gc Composite (4)

ST Wk. Mem.(2)

Cmp. Know.(2)

Fld. Reas.(2)


Cog. Pr. Spd.(2)

LT Retrieval(2)

Visual. Proc.(2)

Aud. Proc.(2)

ST Wk. Mem.(3)

Cmp. Know.(3)

Fld. Reas.(3)

Qnt. Reas.(2)

Perc. Spd.(2)

Number Facility (2)


Reading, Math and Writing Scholastic Aptitudes (each comprised of a mix of 4 different CHC abilities)

Broad Oral Lang (3)*

Oral Expression (2)

Oral Lang. (2)*

Listening Comp. (2)* (* English or Spanish)

Phon. Cod.(2)

Spd. Lx. Acc.(2)

8 CHC narrowability

clusters

RQPN

VL MS

PCLS

LA


Grw-WGrw-R Gq Gc/Gkn

Phon.-Graph. Know. (2)

Rdg. Cmp.(2)

Other clusters available by combining Standard ACH

tests with select tests from Extended ACH easel (9

tests)

(#) = # tests

Rdg. Cmp.(3)

Rdg. Rate(2)

Bas. Wr. Sk.(2)

Math Pr.Slv.(2)

Acd. Knw.(3)

Clusters available from Standard Ach

easel (11 tests)

(#) = # tests

Bas. Rdg. Sk.(2)

Brd. Rdg.(3)

Rdg. Flu.(2)

Rdg.(2)

Wr. Exp.(2)

Brd. Wr. Lg.(3)

Wr. Lg.(2)

Math Cal.Sk.(2)

Brd. Math.(3)

Math.(2)

Academic Skills (3)

Academic Applications (3)

Academic Fluency (3)

Broad Achievement (9)

Brief Achievement (3) New

3 parallel forms

of standard achievement

battery


5/4/2015

7

Increase the cognitive complexity requirements for selected tests and

clusters to provide greater ecological validity and interpretive


The WJ IV Auditory Processing (Ga) cluster is not your father's Ga measure

WJ IV still has the Oldsmobile Ga (Phonetic Coding) in OL: COG now has more cognitively complex Ga measures

The WJ IV has taken a broader contemporary view of the domain of Ga


5/4/2015

8

Loadings on first unrotated common factor

CommunalityEstimates

Verbal Comprehension Index (VCI) 0.809 0.654

Auditory Processing (Ga) 0.804 0.646

Fluid Reasoning (Gf) 0.804 0.646

Perceptual Reasoning Index (PR) 0.800 0.639

Comprehension-Knowledge (Gc) 0.779 0.607

Short-Term Work. Memory (Gwm) 0.764 0.584

Working Memory Index (WMI) 0.749 0.562

Long-Term Retrieval (Glr) 0.683 0.466

Visual Processing (Gv) 0.604 0.365

Processing Speed Index (PSI) 0.569 0.323

Cog. Processing Speed (Gs) 0.537 0.288

1-factor (unrotated) common-factor solution for WJ IV COG / WISC-IV composite scores (n=173)


Provide new, useful and efficient options for comparing abilities

within and across batteries


5/4/2015

9

New and useful options for comparing abilities (PSW) within and across batteries

•Options for exploring individual strengths and weaknesses acrosscognitive, linguistic, and academic abilities

•A new fluid-crystallized (Gf-Gc) composite for comparison to measures of cognitive processing, linguistic competency, and academic achievement for determination of relative strengths and weaknesses across all domains

•Each battery organized for ease of use, leading with a core set of tests in each battery for use as the predictor pool for calculations that identify relative strengths and weaknesses among administered tests and clusters


Math (Gq)

Reading (Grw)

Writing

(Grw)

Other CHC cog abilities

Oral Lang.

abilities

Gf-GcComposite(predictor)

Possible strength

and weakness

target domains

Conceptual summary of new WJ IV Gf-Gc strength and weakness comparison procedure and options


5/4/2015

10

Retain and advance the psychometric quality that has been associated with the

previous editions of the WJ batteries


Retain a focus on psychometric quality that has been associated with the previous editions of the Woodcock-

Johnson batteries


•By providing a new, large, and nationally-representative standardization sample of the US population (n=7,416);

•By updating items and simplifying test administration and interpretation procedures;

•By augmenting the underlying scaling of speeded tests; and

•By utilizing state-of-the art data collection, test development, and data analytic methods as models to facilitate progress in the field of applied test development.

5/4/2015

11

The WJ IV: New ACH Tests (B1)

Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)


WJ IV ACH tests by CHC factor domains

Grw -Reading

CHC Factors

New or sign. change

Grw -Writing

Gq Gc/Gkm

Letter-Word

Identification

(RD)

Passage Comp.

(RC)

Editing

(EU)

Social Studies

(K0/K2)

Sentence Wr. Fluency

(WA/WS)

Reading Vocabulary

(RC/Gc-VL)

Writing Samples

(WA)

Calculation

(A3)

Humanities

(K0/K2)

Spelling

(SG)

Science

(K0/K1)

Applied Problems

(A3/RQ)

Word Attack

(RD/Ga-PC)

Sentence Rdg. Fluency

(RC/Gc-RS)

Spelling of Sounds

(SG/Ga-PC)

Math Facts Fluency

(A3/Gs-N)

Reading Recall

(RC/Glr-MM)

Word Rdg.

Fluency

(RC/Gs-RS)

Oral Reading

(RD/V)

Number Matrices

(Gf-RQ)


5/4/2015

12

New cluster

Cluster introducedwith WJ III NU or cluster with new combination of tests

New or significantlyrevised tests

Test 8: Oral Reading

A measure of oral reading skill (story reading accuracy and prosody) that contributes to the Reading Fluency cluster

Fluent readers must be accurate readers. Measures fluency of oral expression in connected discourse. Fluent reading requires attention to the grammatical structure of prose. The way to ascertain whether an individual has attained this ability requires listening to him or her read connected discourse

Prosody: In reading, prosody refers to the expressiveness with which a student reads.

5/4/2015

13

Types of reading errors that may occur:

Self-corrections within 3 seconds do not count as errors.

Test 8: Oral Reading

Test 12: Reading Recall

A measure of reading skill that contributes to the Reading Comprehension cluster (reading comprehension and meaningful memory)

Examinee reads short paragraphs or stories and then recalls as much of the story as possible

Can compare to Story Recall in WJ IV COG

5/4/2015

14

Test 15: Word Reading Fluency

A measure of reading skill that contributes to the Reading Rate cluster.

A measure of vocabulary knowledge and semantic fluency

Examinee reads a list of four words and examinee must decide which two "go together"

Nine of the WJ IV Standard ACH

tests have optional Qualitative

Observation Checklists with

percentage base rate information for

informal clinical use

5/4/2015

15

What is cognitive complexity? (C)


CHC factor breadth

Degree of g-loading

Complicated(Does not

necessarily equal)

Factorial complexity


Cognitive complexity

5/4/2015

16

• Increase the information processing demands of the tests within a specific narrow CHC domain.

• Not to be confused with factorial complexity• Design tests that place greater demands on:

• Cognitive information processing (cognitive load)• Greater allocation of key cognitive resources (working

memory or attentional control)• The involvement of more cognitive control or executive

functions

WJ IV cognitive complexity design approach based on work of Lohman & Larkin (2011)


Intelligence Testing Related

Research:Levels of

theoretical reductionism and

explanation

White matter tract organization,

integrity & efficiency

-rate of neural oscillations-neural synchronization-Reaction-time and temporal g-ERP’s (e.g., ABR)

PMA1

T2 T3 T4 T5 T6 T7 T8 T9T1 T12T10 T11

PMA2 PMA3 PMA4 …etc

…etc

G1 G2 G3

…etc

g ?

(Consensus Cattell-Horn-Carroll Hierarchical Three-Stratum Model)

-Human Connectome-Functional brain networks (Bressler & Menon, 2010)

-“Rich club” network hubs-P-FIT model

(Adapted from conceptual distinctions of Earl Hunt, 2011)


Human Connectome.avi

Human Connectome.avi

5/4/2015

17


Approach 1. Increasing the cognitive complexity of a test is often accomplished by making the test a mixed measure of more than one narrow CHC ability (factorially complex mixed CHC measures)

One design objective in the WJ IV was to increase the cognitive complexity requirements for

selected tests and clusters to provide greater ecological validity and interpretive relevance of

the measures.


5/4/2015

18

Most contemporary CHC designed individual tests have focused on developing relatively pure measures of each cognitive ability (mental pulley)

Gf Gwm Ga Gv

Gc Gs Glr

Analogy: Think of general intelligence (g) as a system of relatively independent cognitive abilities (relatively construct “pure” pulleys)

working together to deal with a specific cognitive task load


In this approach a test is designed to be a mixed measure of two (or more) cognitive abilities (mental pulleys; Gf + Gv)

Gf + Gv

Gf + Gv Gwm Gs Ga Glr Gc

Approach 1 to developing cognitively complex tests: Construct factorially complex measures (a system of pulleys from 2 or

more domain functions working in combination).


5/4/2015

19

Gf + Gv

Gf + Gwm + Gc + Gq Ga Glr Gv

Approach 1 example

Wechsler Arithmetic test


One design objective in the WJ IV was to increase the cognitive complexity requirements for selected tests and

clusters to provide greater ecological validity and interpretive relevance of the measures.

Approach 2. A second approach is to increase the complexity of information processing demands of the tests within a specific narrow CHC domain (Lohman & Larkin, 2011; McGrew, 2012). This second form of cognitive complexity, not to be confused with factorial complexity, places greater demands on cognitive information processing (cognitive load), requires greater allocation of key cognitive resources (working memory or attentional control), and invokes the involvement of more cognitive control or executive functions (Arend, Colom, Botella, Contreras, Rubio, & Santacreu, 2003; Jensen, 2011; Lohman & Larkin, 2011; Marshalek, Lohman, & Snow; 1983). This second approach to increasing test cognitive complexity was a primary design principle for the WJ IV.


5/4/2015

20

Ga Gv Gwm Gs Gf Glr Gc

Approach 2 is to increase the complexity of information processing demands of the tests within a specific CHC cognitive functional domain. Tasks are still as relatively pure a measure of the CHC domain as possible but there is a deliberate increase in the number of “mini-pulleys” (cognitive information component complexity) that work together within the CHC domain. This was the primary approach used for certain WJ IV tests.


Motor Power, Speed & Timing

(Gp,Gps)

(feedback loop)

Learning (storage) efficiency (Glr)

Retrieval fluency (Glr)

Sensory & PerceptualSystems

(feedback loop)

Beyond CHC TheoryAdapted from Schneider & McGrew

(2012, 2013)

Visual (Gv)

Auditory (Ga)

Tactile (Gh)

Kinesthetic (Gk)

Olfactory (Go)

Motor Control

(Note: e.g.., Gv, Ga, etc. are not simple visual perceptual or sensory processing but the complexity of visual processing that a person can handle)

Gt = Speed of Elem.Perc. Processing

Cognitive Processing Speed (Gs)

Acquired Knowledge Systems (aka, long-term memory)

Etc. (what)

Etc. (how)

Etc. (what)

Etc. (how)

Motor Sequences

(what)

Motor Sequences

(how)

Grw (what)

Grw(how)

Gc(what)

Gc (how)

Nonverbal (e.g., motor)

Cognitive

Environmental Input

Cognitive performance

Motor performance

© Institute for Applied Psychometrics, Dr. Kevin

S. McGrew, 012314

Includes both tacit andexplicit knowledge systems;

declarative (what) and procedural (how) knowledge

5/4/2015

21

The WJ IV COG is not your father’s intelligence test!

The WJ IV COG GIA is a much more cognitively complex (and high g) measure of intelligence


...most disciplines have a common set of terms and definitions (i.e., a standard nomenclature) that facilitates communication among professionals and guards against misinterpretations. In chemistry, this standard nomenclature is reflected in the ‘Table of Periodic Elements’. Carroll (1993a) has provided an analogous table for intelligence…..

(Flanagan & McGrew, 1998)

© Institute for Applied Psychometrics, Dr. Kevin S. McGrew, 012314

WJ IV “Intelligent” testing (D)

5/4/2015

22

•Relative degree of cognitive complexity:High Medium (M/M) Low

•CHC broad factor loading

•Test name abbreviation

•CHC narrow ability code(s)

Te

Go

All information based on analysis of WJ IV norm

data from ages 6 thru 19

•BIS (modified) content/stimuluscharacteristic

NumSeries

(RQ)

.91 .62 H .80

•g-loading

•Reliability

#

•Median correlations with R, M, W

© Institute for Applied Psychometrics (IAP)Dr. Kevin McGrew 3-28-14

.63 .73 .64

Gc

Ga

Gf

Gwm

Gs

Glr

Gv

.78

.75

.71

.60

.71

.66

.76

OralCmp

(LS)PicVoc

(VL)

.82 .69 M .76 .81 .65 M .82

RpdPcNm

(NA)RetFlu

(FI)

.80 .57 M .42 .85 .51 L .24

SndAwr

(PC)Segment.

(PC)SndBlnd

(PC)

.71 .67 H .52 .93 .60 M.74 .88 .53 L .62

SenRep

(MS/LS)

.83 .60 M .48

UndDir

(WM)

.86 .66 M .64

CO

G C

HC

clu

ster

g-l

oa

din

gs

NumSeries

(RQ)

ConFrm

(I)

AnlSyn

(RG)

.91 .62 H .80 .92 .62 M .62.90 .65 M .66

OralVoc

(LD/VL)GenInfo

(K0)

.89 .74 H .86 .84 .59 M .78

StryRec

(MM)VisAudLrg

(MA)

.93 .58 M .54 .96 .52 L .48

LetPtMat

(P)

.90 .55 M .77

Visual.

(Vz)PicRec

(MV)

.83 .60 M .70 .71 .47 L .50

PhnProc

(PC)NonWrRep

PC,UM/MS

.83 .75 H .59 .90 .58 M .18

VerbAttn

(WM/AC)

.86 .65 H .76

.15

.18

.50

.29 .26

.18 .30

.29 Gq

.32 GrwCOGACH

OL

.63 .73 .64 .44 .47 .35 .25 .43 .34

.42 .42 .50

.31 .39 .42 .30 .29 .30

.56 .51 .49 .35 .38 .24

.38 .49 .38 .36 .25 .36

.52 .51 .53 .34 .21 .41

.43 .35 .41 .45 .43 .39

.35 .37 .35 .34 .22 .22

.52 .42 .48 .40 .32 .33

.56 .48 .55 .49 .42 .44 .27 .31 .28

MemWrd

(MS)

.82 .63 M .58

.27 .28 .32

ObNmSq

(WM)

.89 .71 M .74

.32 .42 .35

#

NumRev

(WM/AC)

.86 .61 M .36

.41 .44 .42

#

PairCan

(P/AC)

.89 .49 L .60

.30 .44 .28

.46 .55 .48

NumPtMat

(P)

.84 .53 M .80

.53 .57 .61

Matrix Reasoning (I)Figure Weights (RQ)Picture Concepts (I)Arithmetic (RQ; Gq)

Similarities (VL/LD)Vocabulary (VL)Information (K0)Comprehension (LD/K0)

Block Design (Vz)Visual Puzzles (Vz/SR?)

Digit Span (MS,WM)Letter-Number Seq. (WM)Picture Span (WM/MS; Gv-MV?)

Coding (R9/MA?)Symbol Search (P/R9; Gv-SS?)Cancellation (P/R9)

Naming Speed Literacy (NA)Naming Speed Quantity (NA)Immediate Symbol Translation (MA)Delayed Symbol Translation (MA)Recognition Symbol Translation (MA)

WISC-V tests & tentative CHC

classifications (based on multiple

sources)

No Ga tests

© Institute for Applied Psychometrics (IAP), Dr. Kevin McGrew, 04-29-15

5/4/2015

23

WJ IV Tests of Cognitive Ability:Overview + (D)

Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)


Gf

Gc Gwm

Glr

Gv

Ga

Gs

GqGrw

g

I am going to be your guide for a “walk in the clouds” of human

cognitive abilities

The CHC model will be our map

The WJ IV COG (and OL)batteries will be our measures


5/4/2015

24

Oral Vocabulary

General Information

Number Series

Concept Formation

Verbal

Attention

Numbers Reversed

Story Recall

Visual-Auditory Learning

Visualization

Picture Recognition

Letter-Pattern

Matching

Pair

Cancellation

Phonological

Processing

Nonword

Repetition

Obj-Num

Sequencing

Analysis-Synthesis

Memory for Words

Number-Pattern

Matching

The final 18 WJ IV COG tests by CHC domain

Gc Glr Gv Ga GsGwmGf


WJ III test

New test

WJ III COG, DS or ACH test with new twist

Perc SpdP (2)

Qnt ReasRQ (2)

VocabVL (2)

List AblLS (2)

AMemSpMS (2)

SpLex AcLA (2)

Num FacN (2)

Phn CodPC (2)

Narrow CHC ability clusters (8)

Aud Proc(2)

LT Ret(2)

VisualProc (2)

Cog ProcSpeed(2)

Fld Reas(2)

CmpKnw(2)

ST Work Mem (2)

ST Work Mem (3)

CmpKnw(3)

Fld Reas(3) Broad CHC ability clusters (10)

Gf Gs Ga Glr GvGwmGc

WJ IV CHC broad and narrow ability clusters


5/4/2015

25

Five primary design principles for WJ IV COG GIA

The WJ IV COG GIA cluster tests should:

1. Be the best factor indicators of each CHC broad domain

2. Be the best predictors of achievement from each CHC broad domain

3. Be the most cognitively complex indicators from each CHC broad domain

4. Be the best measures of g (general intelligence) from each CHC broad domain

5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural)


WJ IV COG general ability clusters

BIA cluster

General Information

Concept Formation

Gf-Gc cluster



WJ III test

New test

WJ III COG, DS or ACH test with new twist

Oral Vocabulary

Number Series

Verbal

AttentionStory Recall Visualization

Letter-Pat Match

Phon

Processing

GIA cluster g

5/4/2015

26

WJ IV COG GIA cluster

.77 .76 .68 .63 .60 .75 .61

GIA test g-loadings (PCA: ages 6-19)



Oral Vocabulary

Number Series

Verbal


Letter-Pat Match

Phon

Processing

GIA cluster g

WJ IV COG GIA cluster



Oral Vocabulary

Number Series

Verbal


Letter-Pat Match

Phon

Processing

GIA cluster g

.18 .17 .14 .12 .12 .17 .11

Median test g-weights for GIA cluster

5/4/2015

27

GIA -Standard

GIAOral Vocabulary

General Information

Number Series

Concept Formation

Verbal

Attention

Numbers Reversed

Story Recall

Visual-AudLearning

Visualization

Picture Recognition

Letter-Pat. Match.

Pair

Canc.

Phon.

Processing

Nonword

Repetition

Primary WJ III and WJ IV COG tests and Clusters

Tests in WJ IV COG clusters

Tests in WJ III COG Clusters

WJ IV GIA

WJ III GIA-Standard

* Visual Matching is renamed Number-Pattern Matching in WJ IV

Comparison of composition of primary WJ III and WJ IV COG CHC and GIA clusters

(WJ III Spatial Relations is half of WJ IV Visualization)

(WJ IV Oral Vocab. was part of WJ III Verbal

Comp.)

Auditory Attention

Decision

Speed

Analysis-Synthesis

Retrieval

Fluency

Spatial

Relations

Verbal Comp.

Memory for Words

Sound

Blending

Visual

Matching *

Gc Gwm GsGaGlr GvGf

g


WISC-IV WAIS-IV WPPSI-III KABC-II SB-5 DAS-II

FS IQ FS IQ FS IQ FCI FS IQ GCA

(n =174) (n =177) (n = 99) (n=50) (n = 50) (n = 49)

WJ IV g-measures

General Intellectual Ability (GIA)

0.86 0.84 0.72 0.77 0.80 0.83

Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79

Gf-Gc Composite 0.83 0.78 0.71 0.82

Note. Correlations in italic represent correlations with a pseudo-WJ IV GIA score computed from 7

WJ IV tests (WJ IV COG does not provide an GIA-Edv cluster)

Select concurrent validity evidence: Correlations of WJ IV primary COG g-scores

with external measures

Conclusion: The WJ IV GIA, BIA and Gf-Gc composite clusters demonstrate strong validity evidence as measures of general intelligence when the criterion are the global composite/total

scores from other major IQ batteries in the field

WJ III GIA other IQ score correlations were

from .67 to .76

5/4/2015

28

Arithmetic 0.82Vocabulary 0.81Similarities 0.77Matrix Reasoning 0.75Comprehension 0.71Block Design 0.69Letter-Number Seq. 0.66Picture Concepts 0.63Digit Span 0.62Symbol Search 0.58Coding 0.41

Oral Vocabulary 0.79Phonological Processing 0.79Verbal Attention 0.71Number Series 0.69Story Recall 0.55Visualization 0.51Letter-Pattern Matching 0.37

WISC-IV FS tests latent g-factor

WJ IV GIA tests latent g-factor

(n =174)

Latent gr = 1.0+

(Observed score r = .86)


Loadings on first unrotated common factor

CommunalityEstimates

Verbal Comprehension Index (VCI) 0.809 0.654

Auditory Processing (Ga) 0.804 0.646

Fluid Reasoning (Gf) 0.804 0.646

Perceptual Reasoning Index (PR) 0.800 0.639

Comprehension-Knowledge (Gc) 0.779 0.607

Short-Term Work. Memory (Gwm) 0.764 0.584

Working Memory Index (WMI) 0.749 0.562

Long-Term Retrieval (Glr) 0.683 0.466

Visual Processing (Gv) 0.604 0.365

Processing Speed Index (PSI) 0.569 0.323

Cog. Processing Speed (Gs) 0.537 0.288

1-factor (unrotated) common-factor solution for WJ IV COG / WISC-IV composite scores (n=174)


5/4/2015

29

Fluid Reasoning (Gf) 0.78

Auditory Processing (Ga) 0.77

Short-Term Working Memory (Gwm) 0.76

Comprehension-Knowledge (Gc) 0.74

Long-Term Retrieval (Glr) 0.72

Visual Processing (Gv) 0.68

Processing Speed (Gs) 0.62

© Institute for Applied Psychometrics; Kevin

McGrew 01-23-14

Comparison of WJ III and WJ IV CHC cluster g-loadings in respective norm samples (first unrotated principal component)

Fluid Reasoning (Gf) 0.79

Comprehension-Knowledge (Gc) 0.79

Long-Term Retrieval (Glr) 0.78

Auditory Processing (Ga) 0.69

Short-Term Working Memory (Gwm) 0.69

Processing Speed (Gs) 0.62

Visual Processing (Gv) 0.60

Note changes in relative g-loadingsfor Ga and Gwm

Oral Vocabulary

General Information Gc

Similarities (VL/LD)Vocabulary (VL)Information (K0)Comprehension (LD/K0)

Picture Vocabulary

Comp-Know

Comp-Know: Ext

Vocabulary

WJ IV Oral Vocabulary (COG) 0.89

WISC-IV Vocabulary 0.88WISC-IV Information 0.86WJ IV General Information (COG) 0.82WISC-IV Similarities 0.81WISC-IV Comprehension 0.72WJ IV Verbal Analogies (ECAD) 0.45

Select Comp-Know correlations

WISC-IV VCI = .79

WAIS-IV VCI = .74

KABC-II Gc = .82

(GIA)

OralVoc

(LD/VL)GenInfo

(K0)

.89 .74 H .86 .84 .59 M .78

.56 .51 .49 .35 .38 .24

OralCmp

(LS)PicVoc

(VL)

.82 .69 M .76 .81 .65 M .82

.52 .42 .48 .40 .32 .33

Gc

.71


5/4/2015

30

COG Test 2: Number Series

• Was in WJ III Diagnostic Supplement• Gf-RQ (Quantitative Reasoning)• Not a “controlled learning” test as are Concept

Formation (Gf-I) and Analysis-Synthesis (Gf-RG)• More Gf “in the wild” – without examiner provided scaffolding

• Extensive history as a premier Gf test in thepsychometric measurement of intelligence

• High in cognitive complexity and g. Best single testpredictor of achievement. Best indicator of Gf factor.

• In GIA, BIA, Gf-Gc Composite, Gf, Gf-Ext, QuantitativeReasoning (RQ), and one Math Aptitude clusters.


Number Series

Concept Formation Gf

Analysis-Synthesis

Fluid Reasoning

Fluid Reasoning: Ext

Quantitative Reasoning

Select Fluid Reasoning correlations

WISC-IV PRI = .70

WAIS-IV PRI = .57

KABC-II Gf = .46

(GIA)

Matrix Reasoning (I)Figure Weights (RQ)Picture Concepts (I)Arithmetic (RQ; Gq)

WJ IV Concept Formation (COG) 0.80WISC-IV Matrix Reasoning 0.78WJ IV Number Series (COG) 0.76WISC-IV Picture Concepts 0.63WJ IV Analysis-Synthesis (COG) 0.63WJ IV Verbal Analogies (ECAD) 0.38

Block Design-Gv-VzVisual Puzzles – Gv-VzMatrix Reasoning – Gf-I

Block Design-Gv-VzPic Concepts – Gf-IMatrix Reasoning – Gf-I

Pattern Reasoning - Gf-I/Gv-Vz Story Completion – Gf-RG

NumSeries

(RQ)

ConFrm

(I)

AnlSyn

(RG)

.91 .62 H .80 .92 .62 M .62.90 .65 M .66

.63 .73 .64 .44 .47 .35 .25 .43 .34 #Gf

.78


5/4/2015

31

COG Test 4: Letter-Pattern Matching

• Measure of Gs (perceptual speed) and orthographicprocessing

• This speeded test (all WJ IV speeded tests) is based on a newrate-based method of scaling the scores that eliminates the

need for bonus points

• Within Gs, it matches Number Pattern Matching in g, Gs factor loading, and prediction of achievement. Is

more cognitively complex than Number Pattern Matching

• In GIA, Gs, Perceptual Speed (P), Cog. Eff. and Cog. clusters


Letter-Pattern Matching

Pair CancellationGs

Number-Pattern Matching

Cog Proc Speed

Perceptual Speed

Select Cognitive Processing Speed correlations

WISC-IV PSI = .55

WAIS-IV PSI = .44

(GIA)

Gs

.60LetPtMat

(P)

.90 .55 M .77

PairCan

(P/AC)

.89 .49 L .60

.30 .44 .28

.46 .55 .48

NumPtMat

(P)

.84 .53 M .80

.53 .57 .61

Coding (R9/MA?)Symbol Search (P/R9; Gv-SS?)Cancellation (P/R9)

WJ IV Number-Pattern Matching (COG) 0.82WJ IV Pair Cancellation 0.80WJ IV Letter-Pattern Matching (COG) 0.72WISC-IV Coding 0.62WISC-IV Cancellation 0.57WJ IV Rapid Picture Naming (OL) 0.49WJ IV Retrieval Fluency (OL) 0.43WISC-IV Symbol Search 0.40WJ IV Phonological Processing (COG) 0.20


5/4/2015

32


NonwordRepetition

Ga

Segmentation

Auditory Proc Phonetic Coding

Select Auditory Processing correlations

(GIA)

None available

Ga

.76SndAwr

(PC)Segment.

(PC)SndBlnd

(PC)

.71 .67 H .52 .93 .60 M.74 .88 .53 L .62PhnProc

(PC)NonWrRep

PC,UM/MS

.83 .75 H .59 .90 .58 M .18

.52 .51 .53 .34 .21 .41 .56 .48 .55 .49 .42 .44 .27 .31 .28

Sound Blending

Sound AwarenessNo Ga tests

WJ IV Sound Awareness (OL) 0.84WJ IV Phonological Awareness (COG) 0.75WJ IV Segmentation (OL) 0.71WJ IV Sound Blending (OL) 0.70WJ IV Nonword Repetition (COG) 0.64


COG Test 5: Phonological Processing

• Ga (PC) / Glr (LA/FW)•3 subtests (Word Access; Word Fluency; Substitution• Measures three aspects of speech sound processing that requires the efficient construction of sound-based lexical representations• High in cognitive complexity and g. Best single Ga test predictor of achievement. High loading on Ga and secondary low loading on Gc(accessing the lexicon). Also loaded on narrow LA factor in

broard+narrow bottom-up CFA models.• In GIA, Ga, and all reading and writing scholastic aptitude clusters


5/4/2015

33

Examinee listens to a nonsense word and then must repeat the word exactly.

Requires temporary storage of phonological segments in immediate awareness.

Significant body of research has found such tasks to be significantly related to (and be possible “markers”of) reading disabilities, dyslexia and SLI (specific language impairment)

COG Test 12: Nonword Repetition


OL Test 3: Segmentation

• Ga (PC)

• Examinee listens to words and identifies word parts

• In OL Phonetic Coding (PC) cluster

• Highest loading test on Ga factor across all ages

• A moderate measure of g and predictor of ach. across all ages; much more so (and more cognitively complex) than Sound Blending.

• Such tasks have been reported to be strong predictors of early reading (Bouwmeester et al, 2011; Geuden & Sandra, 2003)

5/4/2015

34

Nonword Repetition

(PC/UM-MS)


(PC/Glr-LA)

Sound Awareness

(PC)

Sound Blending

(PC)

Segmentation

(PC)

Auditory Processing (Ga)

Short Term

Wrk Mem (Gwm)

Most complex

Least complex

COG

OL


An auditory processing battery ?

Story Recall


Glr

Retrieval Fluency

Long-term Ret

(GIA)


Spd of Lex AccessNaming Speed Literacy (NA)Naming Speed Quantity (NA)Immediate Symbol Translation (MA)Delayed Symbol Translation (MA)Recognition Symbol Translation (MA)

WJ IV Visual-Auditory Learning (COG) 0.71WJ IV Story Recall (COG) 0.66WJ IV Retrieval Fluency (COG) 0.29

Glr

.71StryRec

(MM)VisAudLrg

(MA)

.93 .58 M .54 .96 .52 L .48

.31 .39 .42 .30 .29 .30

RpdPcNm

(NA)RetFlu

(FI)

.80 .57 M .42

.35 .37 .35 .34 .22 .22

PPTV4 = .43CELF-4 WM = .57/.14CELF-4 Core = .31/42CASL Core = .42

KABC-II Glr = .64

Select correlations

Long-term Retrieval Speed of Lexical Access


5/4/2015

35

Learning Efficiency

Retrieval Fluency

Meaningful Memory

Associative Memory

Free Recall

Naming Facility

Ideational Fluency

Associational Fluency

Expressional Fluency

Originality

Word Fluency

Figural Fluency

Figural Flexibility

Solution Fluency

Glr

Multiple-Trial and/orDelayed Recall

Glr differs from acquired knowledge systems (Gc, Grw, Gq, Gkn) in that it includes the processes of memory

Major implied Glr division made more explicit based on post-Carroll (1993) research. Learning efficiency (level trait) vs retrieval fluency (rate trait)

Glr: The ability to store, consolidate, and retrieve

information over periods of time measured in minutes,

hours, days and years.

Speed of Lexical Access


Story Recall


Glr

Retrieval Fluency

Long-term Ret

(GIA)


Spd of Lex AccessNaming Speed Literacy (NA)Naming Speed Quantity (NA)Immediate Symbol Translation (MA)Delayed Symbol Translation (MA)Recognition Symbol Translation (MA)

WJ IV Visual-Auditory Learning (COG) 0.71WJ IV Story Recall (COG) 0.66WJ IV Retrieval Fluency (COG) 0.29

Glr

.71StryRec

(MM)VisAudLrg

(MA)

.93 .58 M .54 .96 .52 L .48

.31 .39 .42 .30 .29 .30

RpdPcNm

(NA)RetFlu

(FI)

.80 .57 M .42

.35 .37 .35 .34 .22 .22

PPTV4 = .43CELF-4 WM = .57/.14CELF-4 Core = .31/42CASL Core = .42

KABC-II Glr = .64

Select correlations

Long-term Retrieval Speed of Lexical Access

Learning Efficiency

Retrieval Fluency


5/4/2015

36

Visualization

Picture Recogntion

Gv

Visual Processing

Visual Processing correlations

WISC-IV PRI = .55

WAIS-IV PRI = .57

KABC-II Gv = .37

(GIA)

Block Design (Vz)Visual Puzzles (Vz/SR?)

WISC-IV Block Design 0.78WJ IV Visualization 0.64WJ IV Picture Recognition 0.48WISC-IV Symbol Search 0.34

Block Design-Gv-VzVisual Puzzles – Gv-VzMatrix Reasoning – Gf-I

Block Design-Gv-VzPic Concepts – Gf-IMatrix Reasoning – Gf-I

Rover – Gv-SSTriangles – Gv-Vz

Visual.

(Vz)PicRec

(MV)

.83 .60 M .70 .71 .47 L .50

.38 .49 .38 .36 .25 .36 Gv

.66


Verbal Attention

Numbers Reversed

Gwm

Object Number Sequencing

Short-term Wrk Mem

(GIA)

Memory for Words

WISC-IV WMI = .72WAIS-IV WMI = .67KABC-II Gsm = .42

Select Short-term Working Memory correlations

Gwm.75

SenRep

(MS/LS)

.83 .60 M .48

UndDir

(WM)

.86 .66 M .64

VerbAttn

(WM/AC)

.86 .65 H .76

.42 .42 .50 .43 .35 .41 .45 .43 .39

MemWrd

(MS)

.82 .63 M .58

.27 .28 .32

ObNmSq

(WM)

.89 .71 M .74

.32 .42 .35

NumRev

(WM/AC)

.86 .61 M .36

.41 .44 .42

#

Sentence Repetition

UnderstandingDirections

Memory Span

Digit Span (MS,WM)Letter-Number Seq. (WM)Picture Span (WM/MS; Gv-MV?)

WJ IV Verbal Attention (COG) 0.80WJ IV Numbers Reversed (COG) 0.77WISC-IV Digit Span 0.73WJ IV Letter-Number Seq (COG) 0.69WISC-IV Object-Number Seq 0.65

Number Recall – Gsm-MSWord Recall – Gsm-MS/WM?


5/4/2015

37

COG Test 3: Verbal Attention

• Measure of Gwm (working memory-WM; attentional control-AC)

• More ecological “real world” valid measure of working memory

• High in cognitive complexity and g. Within Gwm, the most cognitively complex, one of best indicators of Gwm factor, and best predictor of achievement

• In GIA, BIA, Gwm, Gwm3, Cognitive Efficiency, and one Reading and 1 Written Language Aptitude clusters.


Low

Degree of AttentionalControl (AC)

Memory for

Words

Sentence Repetition

Memory Span (MS)

Tests

High

Degree of Cognitive Complexity ( Cog. Load)

Working Memory

Capacity (WM)Tests

Numbers

Reversed

Understanding

Directions

Object-Number

Sequencing

Verbal

Attention

HighLow

© Institute for

Applied

Psychometrics;

Kevin McGrew 1-18-15

Via multiple regression,the other five Gwmtests (at ages 6-19)

predicted (R=.70) 49 % of Verbal Attention’s score variance. Aftertaking into account Verbal Attention’s

reliability, approximately 35% of

Verbal Attention’s reliable score variance is not accounted for by

the other five Gwm tests

5/4/2015

38

Organizational overview of the WJ IV variations and comparisons (I)


Comparison Options

• GIA/Achievement

• Scholastic Aptitude/Achievement

• Gf-Gc/Achievement/other cog.-ling. abilities

• Broad Oral Language/Achievement

• Academic Knowledge/Achievement

Five ability/achievement difference score procedures to help compare ability to current levels of achievement

[Procedures account for regression-to-the mean (and how it varies by age)]

(Third method PSW SLD models)


5/4/2015

39

Variation Options

• Intra-cognitive based on COG Tests 1—7

• Intra-achievement • Based on ACH Tests 1—6

• Based on Academic Skills, Academic Fluency, and Academic Applications clusters

• Intra-oral language based on OL Tests 1—4

Four variation procedures to help document an individual’s pattern of strengths and weaknesses.

Based on “core” tests in each battery


(Possible criterion or predicted target scores)

Gf-Gc “hybrid” procedure

Predicted Target Cluster

Score

Regression-based prediction models that account for

regression-to-the-mean

Gf-Gc Composite(Predictor score)

GcGf

WJ IV Gf-Gc cognitive Ability cluster

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency

Academic Applications

Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.-Grp.Kn.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

CHC achievement abilities and WJ IV clusters

Phon. Cod. Sp. Lx. Acc.

ST Wk. Mem.(&Ext)

Cog .Pr.Spd. LT Retrieval Visual Proc.Aud. Proc.

Perc. Spd.

Cognitive Efficiency (& Ext)

Aud. Mm. Sp.

Other CHC broad/narrow cognitive and orallanguage abilities and WJ IV clusters

Gwm Gs Ga Glr Gv

Number Facility

(equals)

SD and PR for calculateddifference score

Difference Score

(Compare to distribution of difference scores in WJ IV norm sample to determine significant

strength or weakness)

Actual Target Cluster Score(minus)

Back to comparison menu

5/4/2015

40

Gf (Other): Average of other (non-Gf) 6 core tests

GcGf

NumberSeries

Gwm Gs Ga Glr

OralVocabulary

VerbalAttention

Letter-Pat.Matching

Phono.Processing

StoryRecall

Visual-ization

PredictedNum. Series

NumberSeries

-(minus)

=(equals)

DifferenceScore

SD and PR for calculated difference scores

(Compare difference score to distribution of difference

scores in WJ IV norm sample to determine significant strength or weakness)

ConceptFormation

Analysis-Synthesis

FluidReasoning

FluidReason-Ext

Quant.Reasoning

NumberMatrices

Other Gf tests Other Gf clustersThe same Gf (Other) score is used to evaluate obtained-predicted difference scores for any other optional Gf test or cluster that is available after testing.

Procedure is repeated for each CHC domain—each CHC core test removed and relevant “other” average computed, etc.

Similar procedures available for OL and ACH based on each batteries core tests and optional tests and clusters.

OralVocabulary

VerbalAttention

Letter-Pat.Matching

Phono.Processing

StoryRecall

Visual-ization

Core cognitive tests for intra-cognitive variation procedure

Explanation of WJ-IV intra-variation procedure: Cognitive example


Gv

General Intellectual Ability (GIA) cluster (Predictor score)

General Intellectual Ability (GIA) /Achievement comparison procedure

Gwm Gs Ga Glr GvGcGf

CHC Cognitive Abilities and WJ IV COG and OL Clusters

Regression-based prediction models that account for regression-to-the-mean (and how it varies by age)

Predicted Target

Cluster Score

Actual Target Cluster Score(minus) (equals)

Difference Score

SD and PR for calculated difference score

(Compare to distribution of difference scores in WJ IV

norm sample to determine significant strength or

weakness)

Oral Language

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency


Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.-Grp.Kn.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

(Possible criterion or predicted target scores)Broad Oral Language

Oral Expression

Listening Comprehension

CHC Achievement Abilities and WJ IV

clusters

Academic Knowledge

5/4/2015

41

CHC Cognitive Abilities and WJ IV COG and OL Clusters


Oral Language/Achievement comparison procedure

SD and PR for calculated

difference scores

(Compare to distribution of difference scores in WJ IV

norm sample to determine significant strength or

weakness)

Broad Oral Language *

(Predictor score)

Sp. Lx. Acc.Phon. Cod.


Regression-based prediction models that account for

regression-to-the-mean (and how it

varies by age)Predicted

Target Cluster Score


Difference Score

Academic Skills

Academic Fluency


Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.-Grp.Kn.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

Reading Math.

Grw Gq

CHC Achievement Abilities and WJ IV ACH clusters

(Possible criterion or predicted

target scores)

Academic Knowledge(* English or Spanish)

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency


Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

CHC achievement abilities and WJ IV clusters

Academic Knowledge(Predictor

score)

Academic Knowledge/Achievement comparison procedure

Predicted Target Cluster

Score


SD and PR for calculated difference score

Difference Score


(Compare to distribution of difference scores in WJ IV norm sample to determine significant

strength or weakness)


CHC cognitive abilities and WJ IV cognitive and oral language clusters

Sp. Lx. Acc.Phon. Cod.

Regression-based prediction models that account for regression-to-the-mean

5/4/2015

42

The WJ IV Gf-Gc comparison procedure has clear implications for operationalizing this model

Common elements of third-method pattern of strength and weakness (PSW) approach to SLD identification (Flanagan et al.)

WJ IV Referral-focused selective assessment: (J)

“New” WJ IV COG-ACH research-based guidance

Dr. Kevin S. McGrew, PhD

Institute for Applied Psychometrics (IAP)




5/4/2015

43

04/15

WJ IV studies

Cormier, McGrew, & Bulut

(2015a, 2015b; manuscripts submitted for publication)

< .01 = no statistical or practical significant effect.10 to .30 = moderate effect> .30 = strong effect

Effect size rules-of-thumb used in WJ III and WJ IV studies the same

5/4/2015

44

6 7 8 9 10 11 12 13 14 15 16 17 18 19Gf WJ III

WJ IVGc WJ III

WJ IVGwm WJ III

WJ IVGlr WJ III

WJ IVGa WJ III

WJ IVGv WJ III

WJ IVGs WJ III

WJ IV

WJ III/WJ IV COG CHC-Basic Reading Skills relations summary and comparison by age

Not sign.

Moderate

Strong


WJ III/WJ IV COG CHC-Reading Comprehension relations summary and comparison by age

Not sign.

Moderate

Strong

6 7 8 9 10 11 12 13 14 15 16 17 18 19Gf WJ III

WJ IVGc WJ III

WJ IVGwm WJ III

WJ IVGlr WJ III

WJ IVGa WJ III

WJ IVGv WJ III

WJ IVGs WJ III

WJ IV© Institute for Applied Psychometrics; Kevin S. McGrew 05-01-15

5/4/2015

45

6 7 8 9 10 11 12 13 14 15 16 17 18 19Gf WJ III

WJ IVGc WJ III

WJ IVGwm WJ III

WJ IVGlr WJ III

WJ IVGa WJ III

WJ IVGv WJ III

WJ IVGs WJ III

WJ IV

WJ III/WJ IV COG CHC-Math Calculation Skills relations summary and comparison by age

Not sign.

Moderate

Strong


6 7 8 9 10 11 12 13 14 15 16 17 18 19Gf WJ III

WJ IVGc WJ III

WJ IVGwm WJ III

WJ IVGlr WJ III

WJ IVGa WJ III

WJ IVGv WJ III

WJ IVGs WJ III

WJ IV

WJ III/WJ IV COG CHC-Math Problem Solving relations summary and comparison by age

Not sign.

Moderate

Strong


5/4/2015

46

OralCmp

(LS)

.82 .69 M .76

.52 .42 48

SndAwr

(PC)Segment.

(PC)SndBlnd

(PC)

.71 .67 H .52 .93 .60 M.74 .88 .53 L .62

.56 .48 .55 .49 .42 .44 .27 .31 .28

AnlSyn

(RG)

.92 .62 M .62

.25 .43 .34

SenRep

(MS/LS)

.83 .60 M .48UndDir

(WM)

.86 .66 M .64

.43 .35 .41 .45 .43 .39

ObNmSq

(WM)

.89 .71 M .74

.32 .42 35

MemWrd

(MS)

.82 .63 M .58

.27 .28 .32

Manifest (actual score) variables

Latent factor variables

WJ IV COG/OL-Basic Reading Skills relations extant research summary


NumSeries

(RQ)

ConFrm

(I)

.91 .62 H .80 .90 .65 M .66

.63 .73 .64 .44 .47 .35#

Basic Reading Skills Aptitude

PhnProc

(PC/LA)NonWrRep

PC,UM/MS

.83 .75 H .59 .90 .58 M .18

.52 .51 .53 .34 .21 .41

GenInfo

(K0)

.84 .59 M .78

.35 .38 .24

OralVoc

(LD/VL)

.89 .74 H .86

.56 .51 .49

PicVoc

(VL)

.81 .65 M .82

.47 .42 .41

Vocabulary(r=.89 with Gc)

When Vocabulary cluster is used instead of Gc cluster, it has a stronger

association with BRS.

LetPtMat

(P)

.90 .55 M .77

.46 .55 .48

NumPtMat

(P)

.84 .53 M .80

.53 .57 .61

PerceptualSpeed

(r=.85 with Gs)

When Perceptual Speed cluster is used instead of Gs

cluster, it demonstrates a significant association with

BRS. Gwm association decreases.

Gc

Ga

Gf

Gwm

Gs

Glr

Gv

Word

Attack

VerbAttn

(WM/AC)

.86 .65 H .76

.42 .42 .50

NumRev

(WM/AC)

.86 .61 M .36

.41 .44 .42

#

OralCmp

(LS)

SndAwr

(PC)Segment.

(PC)SndBlnd

(PC)

SenRep

(MS/LS)UndDir

(WM)ObNmSq

(WM)

AnlSyn

(RG)

MemWrd

(MS)

NumSeries

(RQ)

ConFrm

(I)

GenInfo

(K0)

PhnProc

(PC/LA)

NonWrRep

(PC,UM/

MS)

VerbAttn

(WM/AC)

OralVoc

(LD/VL)PicVoc

(VL)

NumRev

(WM/AC)

Vocabulary

LetPtMat

(P)NumPtMat

(P)

Gc

Ga

Gf

Gwm

GsPerceptual Speed PairCan

(P/AC)

Primary tests and clusters Possible follow-up tests

WJ IV COG/Basic Reading Skillsresearch-based suggested tests and

clusters


StryRec

(MM)Glr

GvVisual.

(Vz)

5/4/2015

47

SndAwr

(PC)Segment.

(PC)SndBlnd

(PC)

SenRep

(MS/LS)UndDir

(WM)

OralCmp

(LS)

ObNmSq

(WM)

AnlSyn

(RG)

MemWrd

(MS)

NumSeries

(RQ)

ConFrm

(I)

PhnProc

(PC/LA)NonWrRep

PC,UM/MS

VerbAttn

(WM/AC)

OralVoc

(LD/VL)GenInfo

(K0)

PicVoc

(VL)

NumRev

(WM/AC)#

LetPtMat

(P)NumPtMat

(P)

Gc

Ga

Gf

Gs

GlrStryRec

(MM)

PairCan

(P/AC)

Perceptual Speed

Vocabulary

WJ IV COG/OL-Reading Comp.research-based suggested tests

and clusters


Gwm

GvVisual.

(Vz)Primary tests and clusters Possible follow-up tests

SenRep

(MS/LS)UndDir

(WM)ObNmS

(WM)

AnlSyn

(RG)

MemWrd

(MS)

WJ IV COG/OL-Math Calc Skillsresearch-based suggested tests and

clusters


NumSeries

(RQ)

VerbAttn

(WM/AC)

OralVoc

(LD/VL)PicVoc

(VL)

NumRev

(WM/AC)

LetPtMat

(P)

Gc

Gf

Gwm

Gs

Gv

PairCan

(P/AC)

Visual.

(Vz)

Perceptual Speed

Quant. Reas.


NumPtMat

(P)

GenInfo

(K0)OralCmp

(LS)

ConFrm

(I)

PicRec

(MV)

Glr

Ga

StryRec

(MM)

PhnProc

(PC/LA)

5/4/2015

48

SenRep

(MS/LS)UndDir

(WM)ObNmS

(WM)

AnlSyn

(RG)

MemWrd

(MS)

WJ IV COG/OL-Math Problem Solvingresearch-based suggested tests and

clusters


VerbAttn

(WM/AC)NumRev

(WM/AC)

LetPtMat

(P)

Gc

Gf

Gwm

Gs

Gv

PairCan

(P/AC)

Visual.

(Vz)

Perceptual Speed


NumPtMat

(P)

OralCmp

(LS)

ConFrm

(I)

PicRec

(MV)

Glr

Ga

StryRec

(MM)

PhnProc

(PC/LA)

NumSeries

(RQ)

OralVoc

(LD/VL)PicVoc

(VL)

Vocabulary GenInfo

(K0)

History or learning problems in reading since starting school

History of early ear infections

Classroom performance and tested reading shows problems in word recognition and reading fluency/speed

Problems in paying attention in class. Also difficulty staying in his seat

Good in mathematics

Avid chess player

Very social

WJ IV example case study (O)(Patrick – 9 years 1 month old Grade 3.6)


5/4/2015

49

Comprehension-Knowledge (Gc-Ext)Fluid Reasoning (Gf-EXT)

Short-term Working Mem (Gwm-EXT)


Auditory Processing (Ga)Long-term Retrieval (Glr)

Visual-spatial Processing (Gv)

Patrick

9-1 3.6

WJ IV Patrick case study: GIA and broad CHC clusters – normative comparisons



Comprehension-Knowledge (Gc-Ext)Fluid Reasoning (Gf-EXT)

Short-term Working Mem (Gwm-EXT)




Patrick

9-1 3.6



Average (50th percentile rank)

Average range


5/4/2015

50

Patrick

9-1 3.6

Comprehension-Knowledge (Gc-Ext)Oral Vocabulary

General Information

Picture Vocabulary

Fluid Reasoning (Gf-Ext)Number Series

Concept Formation

Analysis-SynthesisQuantitative Reasoning (RQ)

Vocabulary (VL)




Patrick

9-1 3.6

Short-Term Working Memory (Gwm-Ext)

Verbal Attention

Numbers ReversedObject-Number Sequencing

Memory for Words

Sentence Repetition

Auditory Memory Span (MS)



5/4/2015

51

Patrick

9-1 3.6

Cognitive Processing Speed (Gs)Letter-Pattern Matching

Pair CancellationNumber-Pattern Matching

Perceptual Speed (P)

Numbers Reversed


Number Facility (N)



Patrick

9-1 3.6

Auditory Processing (Ga)Phonological Processing

Nonword Repetition

Segmentation

Blending

Phonetic Coding (PC)

VisualizationPicture Recognition




5/4/2015

52

Patrick

9-1 3.6

Story RecallVisual-Auditory Learning




Basic Reading Skills

Reading ComprehensionReading Fluency

Reading Rate

Math Calculation SkillsMath Problem Solving

Basic Writing SkillsWritten Expression

Phoneme-Grapheme Knowledge

WJ IV Patrick case study: Select ACH clusters (no broad or cross-domain)Normative comparisons


5/4/2015

53

The WJ IV provides a a variety of variation and comparison procedures for

understanding an individual’s learning needs and strengths and weaknesses

Five ability/achievement difference score comparison procedures to help compare ability to current levels of achievement

Four variation procedures to help document an individual’s pattern of strengths and weaknesses


(Continued on next slide)

Intra-ACH Variations Procedure (+-1 SD): Patrick case study


5/4/2015

54

Intra-ACH Variations Procedure (+-1 SD): Patrick case study




Reading Rate




Significant ACH strengths/weaknesses: Intra-ACH (Extended) variation procedure (+-1.5 SD) – Patrick case study

Test level weaknesses: Letter-Word Identification; Sentence Reading FluencyTest level strengths: None (Math Facts Fluency possible)


5/4/2015

55

Gf (Other): Average of other (non-Gf) 6 core tests

GcGf

NumberSeries

Gwm Gs Ga Glr

OralVocabulary

VerbalAttention

Letter-Pat.Matching

Phono.Processing

StoryRecall

Visual-ization

PredictedNum. Series

NumberSeries

-(minus)

=(equals)

DifferenceScore

SD and PR for calculated difference scores

(Compare difference score to distribution of difference

scores in WJ IV norm sample to determine significant strength or weakness)

ConceptFormation

Analysis-Synthesis

FluidReasoning

FluidReason-Ext

Quant.Reasoning

NumberMatrices

Other Gf tests Other Gf clustersThe same Gf (Other) score is used to evaluate obtained-predicted difference scores for any other optional Gf test or cluster that is available after testing.

Procedure is repeated for each CHC domain—each CHC core test removed and relevant “other” average computed, etc.

Similar procedures available for OL and ACH based on each batteries core tests and optional tests and clusters.

OralVocabulary

VerbalAttention

Letter-Pat.Matching

Phono.Processing

StoryRecall

Visual-ization

Core cognitive tests for intra-cognitive variation procedure

Explanation of WJ-IV intra-variation procedure: Cognitive example


Gv

WJ IV Patrick case study: Intra-COG (Ext) variation output – select output

This information is presented

in color coded

profiles in the next

slides

Allows for simultaneous

viewing of normative and S/W

information(continues for rest of cognitive tests – not shown here)

5/4/2015

56

Comprehension-Knowledge (Gc-Ext))Fluid Reasoning (Gf-Ext)

Short-term Working Mem (Gwm-Ext)




Patrick

9-1 3.6

Significant broad COG cluster strengths/weaknesses: Intra-Cog (Extended) variation procedure (+-1.5 SD) – Patrick case study



Patrick

9-1 3.6

Comprehension-Knowledge (Gc-Ext)Oral Vocabulary

General Information

Picture Vocabulary

Fluid Reasoning (Gf-Ext)Number Series

Concept Formation

Analysis-SynthesisQuantitative Reasoning (RQ)

Vocabulary (VL)

WJ IV Patrick case study: COG clusters and tests strengths and weaknesses as per the Intra-COG (extended) variation procedure (+-1.5 SD)


(+1.35 SD)

(+1.47 SD)


5/4/2015

57

Patrick

9-1 3.6

Short-Term Working Memory (Gwm-Ext)

Verbal Attention

Numbers ReversedObject-Number Sequencing

Memory for Words

Sentence Repetition

Auditory Memory Span (MS)

(-1.44 SD)



Patrick

9-1 3.6

Cognitive Processing Speed (Gs)Letter-Pattern Matching

Pair CancellationNumber-Pattern Matching


Numbers Reversed


Number Facility (N)

(-1.49 SD)

(-1.44 SD)(-1.49 SD)



5/4/2015

58

Patrick

9-1 3.6

Auditory Processing (Ga)Phonological Processing

Nonword Repetition

Segmentation

Blending

Phonetic Coding (PC)

VisualizationPicture Recognition


(-1.37 SD)

(+1.45 SD)

(+1.48 SD)



Patrick

9-1 3.6

Story RecallVisual-Auditory Learning


WJ IV Patrick case study: CHC COG clusters and tests strengths and weaknesses as per the Intra-COG (extended) variation procedure (+-1.5 SD)


5/4/2015

59

Short-term Working Mem (Gwm-Ext)




Patrick

9-1 3.6

Gf-Gc Composite


Phonetic Coding (PC)Auditory Memory Span (MS)

Number Facility (N)Cognitive Efficiency (Gsm+Gs-Ext)

Significant broad, narrow & clinical COG/ACH strengths/weaknesses: Gf-Gc Composite procedure (+-1.5 SD) – Patrick case study




Reading Rate




Significant broad, narrow & clinical COG/ACH strengths/weaknesses: Gf-Gc Composite procedure (+-1.5 SD) – Patrick case study


5/4/2015

60

Gf-Gc composite

andGIA cluster

ACHcomparison

results

Patrick case study

© Institute for Applied

Psychometrics; Kevin S. McGrew

05-01-15



Reading Rate




WJ IV Patrick case study: Select ACH clusters (no broad or cross-domain)Normative comparisons


5/4/2015

61

Actual Basic Writing SkillsActual Basic Reading Skills

WJ IV Patrick case study: Scholastic Aptitude/Achievement comparison (+-1.5 SD):Basic Reading and Basic Writing Skills

Oral Vocabulary (Gc-LD/VL)

Verbal Attention (Gwm-WM/AC)

Number-Pattern Match. (Gs-P)

Phonological Proc. (Ga-PC/Glr-LA)

Predicted BRS & BWS ACHDiscrepancy PR = 26 SD = -0.64Discrepancy PR = 55 SD = +0.12

Num. Pat.Matching

(P)

Phon.Processing

(PC)

OralVocab.(LV/VL)

VerbalAttention

(WM)


Actual Reading FluencyActual Reading Comprehension

WJ IV Patrick case study: Scholastic Aptitude/Achievement comparison (+-1.5 SD):Reading Comprehension, Reading Fluency & Reading Rate


Concept Formation (Gf-I)

Number-Pattern Match. (Gs-P)


Predicted RC, RF, RRDiscrepancy PR = 58 SD = +0.19

Discrepancy PR = 6 SD = -1.59Actual Reading Rate Discrepancy PR = 5 SD = -1.61

Num. Pat.Matching

(P)

Phon.Processing

(PC)

OralVocab.(LD/VL)

Concept Formation

(I)


5/4/2015

62

Actual Written Expression

WJ IV Patrick case study: Scholastic Aptitude/Achievement comparison (+-1.5 SD):Written Expression


Story Recall (Glr-MM)Number-Pattern Match. (Gs-P)


Predicted Written ExpressionDiscrepancy PR = 85 SD = +1.04

Num. Pat.Matching

(P)

Phon.Processing

(PC)

OralVocab.(LV/VL)

Story Recall(MM)


Actual Math Calculation Skills

WJ IV Patrick case study: Scholastic Aptitude/Achievement comparison (+-1.5 SD):Math Calculation


Number Series (Gf-RQ)Pair Cancellation (Gs-P)

Visualization (Gv-Vz)

Predicted Math Calculation SkillsDiscrepancy PR = 55 SD = +0.14

PairCancel.(P/EF)

Visual.(Vz)

Oral Vocab.(LD/VL)

NumberSeries (RQ)


5/4/2015

63

Actual Math Problem Solving

WJ IV Patrick case study: Scholastic Aptitude/Achievement comparison (+-1.5 SD):Math Problem Solving


Analysis-Synthesis (Gf-RQ)Numbers Reversed (Gwm-WM)

Visualization (Gv-Vz)

Predicted Math Problem Solving

Discrepancy PR = 62 SD = +0.31

Visual.(Vz)

NumbersReversed

(WM)

OralVocab.(LD/VL)

Analysis-Synthesis(RG/RQ)


Gf-Gc composite, GIA & SAPT GIA cluster ACH

comparison results

Patrick case study
