Upload
leesha-roberts
View
2.995
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
Prepared by Leesha Roberts, Instructor II
University of Trinidad and Tobago – Valsayn Campus
Evaluating Learner Success and the Instructional Design
Rationale for using Evaluating The Learner in Instruction Design
OverviewWhat is Assessment and Evaluation?How do they differ?What role does it play in the ID ProcessWhen should learner performance be
assessed?How can assessment be made reliable and
validMatching Assessment to ObjectivesHow does an instructional designer determine
when a learner evaluation has been successful
What is Assessment and Evaluation?What is Assessment?
Procedures or techniques used to obtain data about a learner or a product.
What is Evaluation?The process for determining the success level of
an individual or a product on the basis of dataHow are they different?
Assessment collects of information, while Evaluation is the analysis of the assessment pieces
What is Assessment and EvaluationMeasurement – refers to the data collected
which is typically expressed quantitatively (i.e. Numbers)
Instruments - The physical devices used to collect the data e.g. Rating scales, observation sheets, checklists, objectives tests)
What role does Assessment and Evaluation play in the ID Process
Assessment serves as a pedagogical function to:MeasuringDiagnosingInstructing
Information from Assessment can be used as a secondary function in evaluation.
Developing Performance Measurements
Instructional Designers should be capable of developing:TestsWritten questionnairesInterviewsOther methods of measuring performance
Approaches to Assessment
Cognitive AssessmentAffective AssessmentPsychomotor Assessment
Basic Principles of MeasurementTests that measure what a person has learned
to do are called achievement tests.There are two types of achievement tests
Criterion-referenced tests (CRTs), also known as minimum competency or mastery. Basically it allows everyone to know exactly how well students stand relative to a standard.
Norm-referenced tests (NRTs). Basically these tests are designed to “reliably” select the best performers
Reliability and ValidityWhat is Reliability?
Learner evaluation will provide similar results when it is conducted on multiple occasions.
What is Validity?Determines whether the learners have achieved
the intended outcomes of instruction (based on the intended outcomes of the instruction)
Characteristics of Reliability in TestsReliable tests have:
ConsistencyTemporal Dependency
ConsistencyTo increase the consistency of NRT, developers
simply increase the number of items on the test.To increase the consistency of CRT, assess each
competency a test covers.
Characteristics of Reliability in Tests (Cont’d)
The following are factors that affect the number of items developed when ensuring consistency on a test:
1. Consequences of misclassification2. Specificity of the competency3. Resources available for testing
Characteristics of Reliability in Tests (Cont’d)
Temporal Dependency – each time a test is administered, it should produce similar results.
Characteristics of Validity in TestsThere can be no validity without reliabilityThe performance on a CRT must be exactly
the same as the performance specified by the objective.
Achieving validity is not always straightforward.
Matching Assessment to ObjectivesInstructional objectives are a key element in
the development of effective learner assessment.
A direct relationship must exist between the instructional objectives and the learner assessment.
How can you determine whether the intended outcome of an instructional objective is a change in knowledge, skill or attitude?
Matching Assessment to ObjectivesExample:Read the following sentence and identify the
action.The learner will be able to list the three major
warning signs of a heart attach.The action in the instructional objective is to
list – more specifically, to list the three major warning signs of a heart attack.
Cognitive TestsMeasures acquisition of knowledgePaper and Pencil testsRecitationThe six types of test that apply to cognitive
tasks:Multiple-choice
Matching Assessment to Objectives
True-falseFill-in,MatchingShort answeressay
Performance TestsMeasures a student’s ability to do something.There are five types of Performance
Assessment:Performance (examination of actions or
behaviours, that can be directly observed)
Performance TestsProcess (learning ways of doing things such as
problem solving and discussing)
Product (outcome of a procedure is the product which is evaluated against a standard.)
Performance Tests
Portfolios (provides the basis for a product and process review)
Projects (a product assessment which is an in-depth investigation of a topic worth learning more about, according to Katz(1994)
Performance
Process
ProductPortfolios
Projects
Authentic AssessmentFocus is on “real” tasksAchieves validity and reliability by
emphasizing and standardizing the appropriate criteria for scoring such (varied) Products
“test validity” depends on whether the test simulates real-world tests of ability
Involves “ill-structured” challenges and roles that help students rehearse for the complex ambiguities of the “game” of adult and professional life.
Attitudinal Tests
Appropriateness of ItemsHow do you go about writing valid criterion
items?The taxonomies of objectives include sample
test items that can be used as models.
Appropriateness of ItemsAn assessment procedure may be more
appropriate for some learning outcomes than others.
There are several bases on which the logical consistency between assessment and other aspects of design can be determined
Appropriateness of Items
• These are:Matching the objectives to the criterionMatching the type of assessment to the type of
learningMatching the data collection method to the
purpose of the assessment
A successful learner evaluation provides:Data for instructional interventionData as to whether the learner has met the
instructional objectivesRecommendations based of data gathered
Determination of the success of Learner Evaluation