Why assess?

• To support and improve student learning

• To test learning outcomes

Assessment – supporting student

learning (1)

• Quantity and distribution of student effort

• Condition 1 Assessed tasks capture sufficient study time and effort

• Condition 2 These tasks distribute student effort evenly across

topics and weeks

• Quality and level of student effort

• Condition 3 These tasks engage students in productive learning

activity

• Condition 4 Assessment communicates clear and high expectations

to students

• Quantity and timing of feedback

• Condition 5 Sufficient feedback is provided, both often enough and

in enough detail

• Condition 6 The feedback is provided quickly enough to be useful to

students

Assessment – supporting student

learning (2)

• Quality of feedback

• Condition 7 Feedback focuses on learning rather than on marks or

students themselves

• Condition 8 Feedback is linked to the purpose of the assignment

and to criteria

• Condition 9 Feedback is understandable to students, given their

sophistication

• Student response to feedback

• Condition 10 Feedback is received by students and attended to

• Condition 11 Feedback is acted upon by students to improve their

work or their learning

Source: Gibbs, G and Simpson, C. (2004). Conditions under which assessment supports students'

learning. Learning and Teaching in Higher Education vol.1 pp.3-31.

eAssessment – overview

3

21

Key players 4

Key issues

Types of eAssessment

Question & Test Design

Online Tests

Pros & Cons

eAssessment at PolyU

eAssessment – definitions (1)

• The Joint Information Systems Committee

(JISC) defines eAssessment as:

the end-to-end electronic assessment processes where

ICT is used for the presentation for assessment activity,

and the recording of responses.

• The Scottish Centre for Research into On-Line

Learning and Assessment (SCROLLA) (2006)

defines eAssessment as:

an entirely automated process of delivering and marking

assessments using web or intranet resources.

eAssessment – definitions (2)

• The University of Oregon defines eAssessment as:

the use of digital information and communication

technologies to gather and analyse information from

multiple and diverse sources to develop a deep

understanding of what students know, understand, and

can do with their knowledge as a result of their

educational experiences

Questions to think about…

• Do you recognise eAssessment as a

separate category of assessment?

• Is eAssessment just about multiple choice?

• Can eAssessment assess learning levels

appropriate to study in HE?

• Can eAssessment be used to assess relevant

knowledge and skills?

• Can I use (more) eAssessment in my work?

Activity 1

What do you think are some of the main

eAssessment issues for the key players?

Key Issues for Organisations,

Qualification Boards & Governments

• What should be assessed online – high stake? Should it be institution/country-wide?

• National frameworks and guidelines• JISC (E-Learning Framework, Good E-Assessment Practice guide),

Higher Education Academy, EDNA, Australian Learning & Teaching Council Fellowship e-Assessment, European - Leonardo da Vinci project

• Institutional projects, policies and codes of practice

• Dundee University, Open University (UK), Deakin University, University of Adelaide

• Can the institution handle the necessary academic cultural change?

1

Key Issues for Organisations,

Qualification Boards & Governments1

Source: www.scotland.gov.uk

Key Issues for Organisations,

Qualification Boards & Governments

• 7 million adults lack functional numeracy and 5 million

lack functional literacy. The proportion of adults aged

25 - 64 with low or no qualifications is more than

double that in countries like Sweden, Japan and

Canada.

Source: Leitch Report, 2007

• ….up to half of the 12 000 recruits entering the Army

each year have literacy or numeracy skills at levels at

or below those expected of a primary school leaver.

1

Key Issues for Learning Designers,

Publishers & Developers

• Enhance the validity & quality of assessment

• Promote good practice in assessment

• Marking & Grading assessments (in LMS)

• (Intelligent) Feedback

• Engage learners and teachers in assessment

• Equity and diversity of assessment resources (e.g. gender, language, computer anxiety)

• Improve cost-effectiveness of assessment

• Keep up-to-date with latest technology and pedagogical developments in assessment

• Design challenges

2

eAssessment – design challenges

• symbolic systems

• free text processing

• computer-assisted validation of human

marked scripts

• mathematical input

• diagrammatic reasoning

• free-hand graphical input

Key Issues for teachers, authors &

subject matter experts

• What to assess – type? problem-based? project-based?

• Administration within LMS (+ Security)

• Marking & Grading assessments

• Question content – paper-base to electronic

• Will it help to meet learning outcomes?

• Integration into curriculum

• Lack of familiarity with the medium

• Perceptions – just multiple choice?

• Will it ease the workload?

• Will it help the weaker learners (remedial)?

3

Key Issues for learners & other

stakeholders

• Why am I being assessed? Will it help me get my degree?

• Are the right skills being tested?

• Is it good preparation for the “real” exam?

• Will it help to meet the learning outcomes?

• Quick, accurate & detailed feedback

• Security

• Remedial work

• Self-assessment

• Graduate attributes

4

eAssessment – a rationale?

• Many assessment strategies focus on what is easier to measure rather than what is important -measuring lower level cognition

• But if we want students to have 21century skills we must focus on higher order outcomes.

Who decides? The institution? Individual departments? Teachers? Learners?

eAssessment - types

diagnostic tests student’s prior knowledge

formative

(self-

assessment;

peer

assessment)

summative assessment which counts towards

final grade.

adaptive

assessment to promote learning by

providing feedback. Usually not

counted towards final grade.

students have more control over the

authored assessment.

personal response

systemquick way to assess understanding

Assessment – part of the learning

Journey

Activity 2

• How many kinds of activities can be

eAssessed?

• How do you assess?

• What criterion do you use when assessing

your activities?

Forms of eAssessment

Table: Forms of eAssessment

(adapted from a study by Peter Donnan, University of Canberra: Conducting assessment online. (2004)

Form of assessment Examples

Traditional assessment submitted online Essays, reviews, reports, case studies

Automated assessment Multiple choice, short answer, matching,

calculations

Automated assessment – advanced options Multiple choice, short answer,

matching/label matching

Calculations/randomly generated

answers, drag & drop

Invigilated online exams – (mid/final semester) Range of formats, multiple choice/ short

answer, automated

Longer essay type

Online discussions Forums, online debates/role plays,

invited online guests

Group projects PowerPoint presentations, CD-ROMs,

multimedia projects, group online

projects laboratory reports, networked

collaborative learning, wikis

Authentic assessment Simulations, critical incident analysis,

case studies, access to external databases

Critical reflection and meta-cognition Electronic portfolios, online journals,

blogs, embedded reflective activities

Advanced problem-solving Problem-based learning scenarios,

learning contracts

Assessment Methods &

Learning Outcomes

• Matching learning outcomes with assessment

methods

Learning Outcomes

•Reflect

•Evaluate

•Predict

•Argue

•Apply

•Relate

•Explain

•Describe

•Compute

•Identify

•Recall

Assessment methods

•Portfolios

•Reflective writing

•Practicum/workplace assessment

•Group projects

•Problem or case-based assessment

•Extended essay

•Essay exam

•Objective test (e.g. MCQ)

Higher level

Thinking

Question & Test Design –

more than just multiple choice!

• Objective Tests

• Question types

Online tests

• PolyU examples

• Examples

• eAssessment resources

eAssessment @ PolyU

• Moodle – English Language

• Clinical Education

• Anatomy

eScholars Group

Mobile-based Glossary Tool

Statistics Simulators

Self-test Program

Examples

• PRINCE 2

• English Language Exams

• OpenMark – formative

• OpenMark – diagnostic

Online testing – planning process

Source: Sheffield Hallam University, UK

eAssessment Resources (1)

eAssessment Resources (2)

eAssessment – other resources

• QuestionMark Perception

• Pearson

• Maple TA

Activity 3

• Can you think of any advantages and

disadvantages of eAssessment?

eAssessment - pros

• Immediate feedback can be given to students

and to staff

• Assessments can be linked to other online

course materials being used

• Ease of marking/consistency/accuracy

• On demand delivery

• Geographic flexibility

• Time flexibility/convenience

• Cost effectiveness

• Ease of administration

eAssessment – cons

• Staff need (more) time and extra skills to create eAssessment resources

• Testing higher order skills requires greater effort

• More difficult to assess written expression or creativity

• Restrictions of multiple choice

• Accessibility

• Authentication/candidate validity

• Security/confidentiality

• Cost

• Technical failure/issues & lack of support

Questions to think about

• Do you recognise eAssessment as a

separate category of assessment?

• Is eAssessment just about multiple choice?

• Can eAssessment assess learning levels

appropriate to study in HE?

• Can eAssessment be used to assess relevant

knowledge and skills?

• Can I use (more) eAssessment in my work?

Goodbye!!!

Jenny

etjeni03@inet.polyu.edu.hk

Ian

etian@inet.polyu.edu.hk

Contact us: