Upload
eduworks
View
99
Download
0
Tags:
Embed Size (px)
DESCRIPTION
In the age of information, only personalized learning and education enables the adaption to always changing requirements. In health care, labor forces are especially in a continuous need of improvement and adaption. Maintaining an optimal care sets a dual goal in the education of health professionals: adapt to scientific developments while retaining the compliance with changing laws and regulations. Computerized adaptive testing (CAT), as a dynamic approach to education, is providing here the right adaptivity by optimizing time and precision of learning and scaling to the ability of the learner. This paper will provide an overview on the current state of CAT and connected methodologies and shed light on the potentials for the next generation of adaptive testing, which can support the emergence of novel ways of education. Its strength is shown on a scenario from the health care sector.
Citation preview
- 24
Extending Computerized Adaptive Testing to Multiple Objectives:Envisioned on a Case from the
Health Care.Christian Weber, Réka Vas
Egovis 2014
- 24
Background: The eduworks network
2023.04.12. Corvinno Technology Transfer Center 2
University of Salamanca
- 24
For a better educational testing
2023.04.12. Corvinno Technology Transfer Center 3
Cognitive Type
Know-ledge
CultureMatching
Labour
Education
Human
profile
Capturing the user
- 24
The health perspective
2023.04.12. Corvinno Technology Transfer Center 4
- 24
Need for ongoing training in the health care
• Immense pressure on doctors within their daily practice and as well in terms of an (expected) ongoing trainingE.g. obligation for training in Germany since 2005
• More and more tasks are transferred to nurses or other caretakerE.g. they have a major role in Alzheimer early detection
(World Alzheimer Report 2011)
• But .. a focus for ongoing training for nurses is missing need for support in educationBut the need comes with special demands..
2023.04.12. Corvinno Technology Transfer Center 5
- 24
The demands of medical training
• Time is restrictedFor education (but also for the support of domain experts)
• The learning has an educational context• Must-know knowledge
regulations & mandatory knowledge
• Should-know knowledgenew insights & specializations
• The learning has a cultural contextCare taker and nurses have different cultural backgrounds with
different perception to learning. patients
2023.04.12. Corvinno Technology Transfer Center 6
- 24
Designing a new test
2023.04.12. Corvinno Technology Transfer Center 7
Learning goal
Know-ledge
Restricted time
CultureInfluenceNurses
Labour
Education
Human
profile
Scaling education to the user
Time to learn
Time to test
- 24
Computerized Adaptive Testing - CAT
2023.04.12. Corvinno Technology Transfer Center 8
pass
pass
fail
passpass
passfail
fail
pass
Te s t I t e m s
Ite
m D
iffi
cu
lty A b i l i t y L e v e l
100
- 242023.04.12. Corvinno Technology Transfer Center 9
Item response function (3PL)as the probability that a person with ability “theta”
answers an item
• b – difficulty, (half way between c(min) and 1)• a – discrimination, (slope at b, differentiating the users)• c – chance for guessing
(building on a logistic function)
- 24
Limitations of question organization
2023.04.12. Corvinno Technology Transfer Center 10
Cognitive Type
Know-ledge
Culture
Q40
Q60
Q60
Q60
Q70
Q60
Q40
Q65
Q70
Q50
- 24
Add Meta Information• Cultural context• Learning type• Obligation yes/no•...
Cognitive Type
Know-ledge
Culture
Q40
Q60
Q60
Q60
Q70
Q60
Q40
Q65
Q70
Q50
Limitations of question organization
2023.04.12. Corvinno Technology Transfer Center 11
- 24
Add Meta Information• Cultural context• Learning type• Obligation yes/no•...
Cognitive Type
Know-ledge
Culture
Q40
Q60
Q60
Q60
Q70
Q60
Q40
Q65
Q70
Q50
Exploring the user on the education
User domain
Educational domain
Limitations of question organization
2023.04.12. Corvinno Technology Transfer Center 12
- 24
Overcoming limitations
• How to grasp the user knowledge in the best possible way?Framing question with no final answer.
• How to add more context to it?Extending the structure and indicators.
• How to detect also his/her understanding of an area?Using knowledge, structured and in relation.
2023.04.12. Corvinno Technology Transfer Center 13
- 24
The demands of medical training - revisited
• Time is restrictedFor education (but also for the support of domain experts)
• The learning has an educational context• Must-know knowledge as meta data
regulations & mandatory knowledge
• Should-know knowledge as meta datanew insights & specializations
• The learning has a cultural contextCare taker and nurses have different cultural backgrounds with
different perception to learning. through modelling
2023.04.12. Corvinno Technology Transfer Center 14
- 24
Differential Item Functioning - DIF
• Occurs when people of different groups have a different probability to answer correct, independent of the same underlying ability.Groups in terms of culture, educational character or learning
style
• Detection normally through statistical analysis of the performance on the test.The designer view: Detection only if a
grouping context is known..
2023.04.12. Corvinno Technology Transfer Center 15
- 24
DIF context through virtualization
• Instead of removing DIF items, redundant item modelling(Makransky and Glas)
• DIF biasingHarder or easier
(difficult level)
• Alternatively:Redundant question
and real structure
But how could a structure look like?..2023.04.12. Corvinno Technology Transfer Center 16
Item Context
12
n
12
nVirtualized Item
- 24
Structure - A new concept of layers
2023.04.12. Corvinno Technology Transfer Center 17
Q11Q12
Q13 Q21Q22
Q31Q32
Q33
Qxy+1
Qxy
Q41Q42
1
2
3
4
y
“meta-parameter”
question ite
ms
sub-ontology grouping
ontology
e.g. item difficulty, cognitive type..
complexity / relations
educational domain
difficulty new ?= f(diff Q xy , compl N x)
++ meta-information
- 24
Event
the system behavior as a stream of data
Agent
Agent
Consumer
Consumer
EventConsumer
PatternDetection
Trans-for-mation
Filter
Agent
Data
Data
Eventproducer
Testing strategy: Thinking about a new frame with (Complex) Event Processing (CEP)
2023.04.12. Corvinno Technology Transfer Center 18
- 24
DomainOntology
A new frame for CAT with context
2023.04.12. Corvinno Technology Transfer Center 19
Meta-
Strategies
TestingStrategies
Data
the system behavior as a stream of data + access to the ontologyData
Eventproducer
IntelligentTest Engine
Event
Q11
Q12
Q13
Q21
Q22
Q31
Q32
Q33
Qxy+1
Qxy
Q41
Q42
1
2
3
4
y
“addon-parameter”
concept group level
ontology
question level
ContextAgent
ContextAgent
ContextAgent
change/modify
e.g right/wrong~perform welltime for answer
DataEvent
e.g learning typecompatible to job xfast tester
EventConsumer
- 24
Exploration, yes - but where (how) to go..
2023.04.12. Corvinno Technology Transfer Center 20
Q QQ
Q
?
?
?
Meta-Strategies
TestingStrategies
IntelligentTest Engine
- 24
Next level questions..
• How are the rules to switch and modify the testing strategy and how to manage it on the fly?..depth/breadth first, related concepts, maintain main test
• What is the best choice to model a context parameters for the strategic exploration?
• What is the best fusion of parameters?.. psychometrics and graph/ontology based complexity
measures
2023.04.12. Corvinno Technology Transfer Center 21
- 24
Data Sources
2023.04.12. Corvinno Technology Transfer Center 22
• Now:• Educational knowledge within the domain ontology as classes
and relations• Logging feedback from the actual testing
• Future:• Target-oriented performance feedback from the user To:
• grouping of users after special criteria• Feedback on the content – e.g. Improving meta-data on the nodes
- 24
Summary
• Need for a crisp testing to detect training potential in knowledge intense professions.Especially in the health care sector
• CAT is a suitable solution for educational testing....but there is room for improvement for a context aware testing
• A new frame and thinking could enable a next generation of context sensitive tests, with..• structured backing knowledge (ontology)• a novel testing behaviour (modifying test behaviour online)
2023.04.12. Corvinno Technology Transfer Center 23
- 24
Thank you for your attention!
Corvinno Technology Transfer CenterH-1093 Budapest, Közraktár utca 12/a.
Tel: +36 1 210 80 62 http://www.corvinno.com