Introduction to CEM and Computer Adaptive Assessments Belfast, March 6 th 2013 Neil Defty Business...

Introduction to CEM and Computer Adaptive

Assessments

Belfast, March 6th 2013Neil Defty

Business & Development ManagerCEM

Neil@cem.dur.ac.uk

Who / What is CEM ?

Centre for Evaluation & Monitoring• Not for Profit Organisation• Part of Durham University• Close links to School of Education• Established 1983• Nursery / Reception Post-16• Monitoring Systems, Research and Evaluation Projects• UK & International• Over 1/3 of UK Secondary & Post-16 establishments• Informed by Evidence from Research

CEM Provides

• Information Systems from Nursery to Post-16 Student Guidance & Chances Performance Monitoring

• Evaluation & Research Projects• Entrance Testing• Support – Phone, email, documentation• Training• INSET• Roadshows• Conferences (including exhibitions)

The Systems

The Systems

Year 7

Year 8

Year 9

MidYIS Computer Adaptive Baseline Test

Year 10

Year 11 Yellis Computer Adaptive Baseline Test

Year 12

Year 13 Alis GCSE Computer Adaptive Baseline Test

GCSE

A / AS / Btec / IB etc

INSIGHT Combines curriculum tests with developed ability

KS3

Typical TimelineMeasure BaselineSTART:

Autumn Term

Feedback

Reports

Collect and Send Results

Value-Added

Feedback

August

END:

September

The baseline score is measured as early as possible and the information sent to CEM

The institution then receives feedback from CEM: baseline test results and predictions

The public examination results are sent to CEM when received by the institution

The value added data is available from September via the Institution’s secure website at CEM

Typical Timeline

Early Testing & Predictions

Testing is available from June at end of previous academic year

e.g. Y6 / Y9 / Y11

Basic Principles:

• Measure Student’s ability Intake Profiles IPR - Strengths and Weaknesses

• Project future performance from national trends Predictions

• Record Final Grade

• Compare actual performance to Projected Performance Value-Added Reports

Baseline Assessment and the Computer Adaptive Baseline Test

(CABT)

Baseline Assessment

• Year 7 (8)• Year 9• Year 10 (11)• Year 12 (13+)

Computer Adaptive Baseline Test

Vocabulary Maths Non Verbal

What is an Adaptive test ?

• All questions allocated to groups of different difficulty

• All students in a year group get common difficulty starting question

• If answer correct, next question harder

• If answer wrong, next question easier

• Test looks at all answers in a section and homes in on questions of a

suitable difficulty for student

Try it yourself at www.intuproject.org/demos

Adaptive approach

Low Average High

X

Standardisation

• Test scores are standardised; Mean = 100, SD = 15

Standardised Score

National Percentage

Comment

>130 Top 2.5% Traditional classification of ‘mentally gifted’

>120 Top 10%

>100 Top 50%

<80 Bottom 10%

<70 Bottom 2.5% Potential special educational needs?

The Computer Adaptive Baseline Test:

• Adaptive components common to MidYIS, Yellis, and Alis are:– Vocabulary– Maths

• Non- adaptive sections:– Non-verbal (MidYIS, Yellis, Alis)– Skills (MidYIS only)

• Assessments are ‘curriculum free’: i.e. students are not taught prior to the test

• Measure of acquired ability• There is no preparation for the test

Difficult vocabulary

Difficult maths

x/4 + 2

Non-Verbal: Cross-sections

Non-Verbal: Blocks

3

PicturesNon-Verbal: Pictures

Skills: Proof Reading

Skills: Perceptual Speed and Accuracy

Alis - Patterns

Alis – Logical Reasoning

Additionally for full INSIGHT:

Developed Ability

• Vocabulary

• Non verbal, Skills

• Attitudinal measures

Curriculum-based assessments covering:

Maths - Number & Algebra- Handling Data- Space, Shape & Measurement

Science - Biology, Chemistry, Physics, - Attitudes to Science

Reading - Speed Reading- Text Comprehension- Passage Comprehension

INSIGHT

INSIGHT - Maths

7

INSIGHT - Maths

Plus Attitude to Science: Perception of Science, Relevance to Self, Environmental Actions, Enjoyment of Science

INSIGHT - Science

INSIGHT – Reading(Text Comprehension)

INSIGHT – Reading(Passage Comprehension)

INSIGHT – Reading(Speed Reading)

INSIGHT – Developed Ability