Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Multidimensional Computerized Adaptive Testing for Rapidly Identifying Patients’
Rehabilitative Care Needs
Chun WangDepartment of Psychology
1
Chun WangDepartment of Psychology
Outline
• Brief introduction to computerized adaptive testing (CAT)
• Multidimensional CAT• “Computerized Adaptive Testing to Direct Delivery of Hospital‐Based Rehabilitation” (NIH R01HD079439, 2015‐2020) – Item bank calibration– Item selection– Stopping rules
• Ongoing projects
2
CAT: basic ideas and advantages
• Adaptive Testing – Binet‐Simon Intelligence Test
• “Computerized Adaptive Testing” (Weiss, 1970s)
• Advantages of CAT– More accurate estimate of the latent traits – Shorter tests – Immediate scoring – Enhanced test security– New item types
3
CAT illustration(Created by ETS 2013, all rights reserved)
4
Early CAT vs. New advancement
• Unidimensional vs. Multidimensional latent traits—Issue of dimensionality in IRT
• Is pain a “unidimensional” construct?
5
6
Multidimensional CAT
• Increased measurement efficiency (Segall, 1996)
7
LifeLife
ChemistryChemistryPhysicsPhysics
Science
CAT: FlowchartAdminister the first item
Estimate
Converged? Or Max. iterations
Select and administer the next item based on
Y
N
Stop YYY
8
, , …,
CAT: FlowchartAdminister the first item
Estimate
Converged? Or Max. iterations
Select and administer the next item based on
Y
N
Stop YYY
Intermediate ability estimation
Item selection algorithm
9
Stopping criterion
Calibration of item pool
Outline
• Brief introduction to computerized adaptive testing (CAT)
• Multidimensional CAT• “Computerized Adaptive Testing to Direct Delivery of Hospital‐Based Rehabilitation” (NIH R01HD079439, 2015‐2020, PI: Dr. Andrea, Y., Cheville) – Item bank calibration– Item selection– Stopping rules
• Ongoing projects
10
An example: “Computerized Adaptive Testing to Direct Delivery of Hospital‐Based Rehabilitation ”
Activity measurePost‐Acute Care (AM‐PAC)
Basic Mobility
Applied Cognition
Daily Activity
Study I
12
Multidimensional IRT Model(Adams, Wilson, & Wang, 1997; Reckase, 2008; Wang & Nydick, 2015)
13
Multidimensional IRT Model(Adams, Wilson, & Wang, 1997; Reckase, 2008; Wang & Nydick, 2015)
• Correlated‐factor simple‐structure
14
Multidimensional IRT Model(Adams, Wilson, & Wang, 1997; Reckase, 2008; Wang & Nydick, 2015)
• Correlated‐factor simple‐structure
15
, 0), 0), 0), 0), 0)
0,0,0,0,0, ,
An Example Item
• How much DIFFICULTY do you currently have reading a long book (over 100 pages) over a number of days?– Unable (0)– A lot (1)– A little (2)– None (3)
16
Multidimensional IRT Model(Adams, Wilson, & Wang, 1997; Reckase, 2008; Wang & Nydick, 2015)
• Polytomous responses– Multidimensional graded response model (Samejima, 1969, Jiang,
Wang, et al., 2015)
17
( )*
'11
kk bP Pe
x k a
* *1k k kP x k P P
a is the discrimination parameter’s (boundary parameter) are ordered from smallest to largest
For x=0,…,K, ∗ 1, ∗
Boundary response function
Illustration of the GRMmodel
18
( )* 1
1 ka bkP Pe
x k
Illustration of the GRMmodel
– Unable (0)– A lot (1)– A little (2)– None (3)
19
Responding “a lot” and above
( )* 1
1 ka bkP Pe
x k
Illustration of the GRMmodel
20
Responding “None” and above.
– Unable (0)– A lot (1)– A little (2)– None (3)
( )* 1
1 ka bkP Pe
x k
21
Responding “None” and above.
* *1k k kP x k P P
Boundary response curve Category response curve
Responding “unable”
“None”
( )* 1
1 ka bkP Pe
x k
Category response curve
22
Unable (0)A lot (1)A little (2)None (3)
Applied Cognition
Prob
ability of cho
osing op
tion k
Category response curve
23
Unable (0)A lot (1)A little (2)None (3)
Applied Cognition
Prob
ability of cho
osing op
tion k
Category response curve
24
Unable (0)A lot (1)A little (2)None (3)
Applied Cognition
Prob
ability of cho
osing op
tion k
Item bank structure
Activity measurePost‐Acute Care (AM‐PAC)
Basic Mobility
Applied Cognition
Daily Activity
• Three dimensional• Polytomous (Likert scale) responses• Simple structure
Item bank
26
• 324 Items in total• 3 domains: Applied cognition (109), Daily activity (106),
Mobility (111)• Minimum sample size requirement: Jiang, Wang, & Weiss
(2016)
• Item selection?
Item information
• Fisher information (Fisher, 1925)
• Observed (category) information
27
11 0
11 1
, , ,
2
2
log ( )( ) X
L XI E
exp
2
2
log ( )( )
L XO
Item observed information
28
Response=0
Item observed information
29
Response=0Response=1
Item observed information
30
Response=0Response=1Response=2
Item observed information
31
Response=0Response=1Response=2Response=3
Item observed information
32
‐2.78 ‐0.63
Response=0Response=1Response=2Response=3
Item Fisher information
33
Item Fisher information
How much DIFFICULTY do you currently have walking on an uneven surface (e.g., grass, dirt road or sidewalk, brick walkways, sidewalks with curb and driveway?) (Mobility)
34
• How much DIFFICULTY do you currently have applying spreads to bread using a knife? (Daily activity)
35
Item information for different items
36