24
Assessment of Higher Education Learning Outcomes Hamish Coates [email protected] acer.edu.au/highereducation

Hamish coates noma tues

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Hamish coates noma tues

Assessment of Higher Education Learning Outcomes

Hamish [email protected]

acer.edu.au/highereducation

Page 2: Hamish coates noma tues

Rationales for change

Building assessment collaboration

s

Building capacity and improvement

Page 3: Hamish coates noma tues

Rationales for change

Page 4: Hamish coates noma tues

What is the “best” university in your country? How do

you know?

Page 5: Hamish coates noma tues

95%

65%

An experiment

Page 6: Hamish coates noma tues

Score distributions for eight groups (institutions, faculties, subjects,

etc.)

Perf

orm

an

ce

Minimum standard

“Standards”, not “standardisation”

Generalising the principles of good assessment will

encourage diversification and

excellence

Page 7: Hamish coates noma tues

Little data

Commitment to

effectiveness

UniversalElite Mass

Happiness data

Effectiveness data

Page 8: Hamish coates noma tues

Data for transparency

Page 9: Hamish coates noma tues

Plan

Act

Evaluate

Improve

Hunch

Data for leadership

Page 10: Hamish coates noma tues

Enhanced management, evaluation and quality monitoring – to do more, better, with less

Higher education growing in significance and scale: clear rationales for increasing output and ensuring quality outcomes

Huge cost and competitive pressures – evidence-based management helps expand provision with quality

Internationalisation pervades all facets of teaching and learning, and graduate outcomes – international perspectives are vital

Shaping complexitie

s

Further reliance on flawed, simplistic rankings will constrain growth and prosperity – robust multidimensional perspectives required

Page 11: Hamish coates noma tues

Building assessment collaboration

s

Page 12: Hamish coates noma tues
Page 13: Hamish coates noma tues

AHELO at a

glance

Are valid and reliable cross-cultural/-institutional/-linguistic comparisons of HE learning outcomes scientifically and practically feasible?

Conceived ~2005, announced 06, scoping 06-09, work 10-13

An international feasibility study – not a ‘pilot’

For students near end of ‘bachelor degree’, also faculty, national managers, institutional coordinators, policymakers, expert advisors, etc.

Design, develop, validate and evaluate frameworks, tests, infrastructure and processes

HEI reports (not for systems), along with international reports and materials, and recommendations for a main study

>300 individuals involved directly

~270 HEIs

Up to 30,000 leaders, faculty, students

Many thousands of public stakeholders

~$30m total cost

Page 14: Hamish coates noma tues

‘Strands’ of testing

Contextual dimension

Generic Skills Economics Engineering

Student Context Instrument

Faculty Context Instrument

Institution Context Instrument

National Context Information

Page 15: Hamish coates noma tues

• Outcome specification

• Document analysis• Consultation• Synthesis, review

Framework creation

• Gather existing materials

• Item workshops• Technical review• Framework mapping• Adaptation,

translation• Verification

Item creation

• Qualitative testing

• Quantitative testing

• OperationalisationInstrument validation

Framework and item

development Tuning AHELO / QAA frameworks Curriculum documents Accreditation systems Discipline research

Authentic, hybrid item types ‘Above content’ reasoning Focus on ‘common core’

Page 16: Hamish coates noma tues

AHELO world map

Linguistically diverse…

Arabic Dutch English Finish Flemish Italian Japanese Korean Norwegian Russian Slovak Spanish

+ Abu Dhabi…

Page 17: Hamish coates noma tues

Building capacity and improvement

Page 18: Hamish coates noma tues

Change horizons

Assessment collaboration

s

Institutional

positioning

Faculty improvemen

t

Policy research

Learner feedback

Stakeholder

engagement

System monitoring

Page 19: Hamish coates noma tues

Building communities

Page 20: Hamish coates noma tues

Building awareness

Page 21: Hamish coates noma tues

Reporting results

http://oecdbetterlifeindex.org

Page 22: Hamish coates noma tues

Types ofsharing

Assessment collaboration

sBuild frameworks

Produce items

Select and use

Analyse and report

Definitions Tasks Processes Results

Page 23: Hamish coates noma tues

How assessment collaboratio

ns work

Pre

para

tion Establish

coordinating centreTranslate and validate instrumentsEngage institutionsTrain institution coordinatorsPrepare for testing

Assessm

en

t

Select studentsAdminister secure testScore responsesVerify and provide data

Rep

ort

ing

Prepare multilevel benchmarking reportsDistribute reports to institutionsInterpretation for monitoring and improvement

Page 24: Hamish coates noma tues

Assessment of Higher Education Learning Outcomes

Hamish [email protected]

acer.edu.au/highereducation