Embedding Quality in Student Assessment in ODL: The case of the University of

Pretoria

Dr Ruth Aluko (University of Pretoria)

&

Dr Ephraim Mhlanga (SAIDE)NADEOSA Conference, 29 – 30 August 2011

Assessment: Type & Purpose

Type Purpose Agency StudentFormative Learning

Teacher

Teacher Certification Summative Student Student Accountability

systemSource: Shavelson (n.d)

Assessment as a monitoring mechanism

Assess diagnose address re-plan

Source: Brenner & Shalem (2010)

An integrative approach to assessment

• Linking assessment task with construct

• Shift from using assessment to measure what was learnt in the past to something that guides teaching and learning.

• From quality control to quality assurance (Leahy et al. 2005 in Brenner & Shalem 2010)

The Questions

• What is the quality of assessment practices of DE programs at the University of Pretoria?

• How far do these agree with the assessment policy of the University on DE programs, international and local quality criteria?

• What are the challenges, and what is the way out?

Rationale• Shift from process of education to outcomes of

student learning for program and institution evaluation (American Psychology Association (APA), 2002)

• Value of assessment to DE students (motivation & retention)

• Dearth in literature on student assessment in distance learning (Hughes, Okumoto & Crawford, 2010)

Background to the StudyDE context at UP• Large scale (20 000+)• Throughout SA (Rural)• 95% Teachers• 68% Female• 70% over 40 years• Internet access: 1% (2007); 14% (2010)

Quality Assurance at UP

• Quality is a priority • Its management is seen as an iterative processIt is always possible to improve• QA is part of a philosophy, a managerial, a

teaching and an administrative style• Its policy with regard to DEAll DE programs must meet the same quality

standards required of CE programs

QA of DE Programs (The Unit for Distance Education)

• University of Pretoria, DE Policy• Guided by:Commonwealth of Learning (COL). 2004. Planning and

Implementing Open and Distance Learning SystemsCouncil on Higher Education (CHE). 2004. Criteria for

Institutional Audits, Higher Education Quality CommitteeCouncil on Higher Education (CHE). 2004. Criteria for

Programme Accreditation, Higher Education Quality Committee

NADEOSA Quality Criteria (As agreed upon by DE providers in SA) (Welch & Reed, 2005)

Instrument for Quality Assurance:NADEOSA Quality Criteria (2005)

1. Policy & Planning2. Learners3. Program Development4. Course Design5. Course Materials6. Assessment7. Learner Support8. Human Resource Strategy9. Management & Administration10. Collaborative Relationships11. Quality Assurance12. Information & Marketing13. Results

Assessment information

Formative & summative Assessment tasks

Assessment of experiential & workplace learning

Assessment-literate & competent staff

Training of part-time tutors

Internal & external moderation

Consistency & accuracy of marking, grading & feedback

Clear procedures to receive, record, process & turnaround

Appeal system

Security of administered summative assessment

Assessment Design

Quality of Assessment

Assessment Management

Security

Certificate integrity

NADEOSA Quality Standards on Assessment & UPUDE Response

Challenges

• Delays in delivery of assignments (from & to students)

• Continual re-training of service providers• Continual monitoring of the quality of their

service• Electronic assignments & electronic marking• Diversifying multiple-mode assessment

Suggestions & Conclusion• Delays are sometimes beyond UP’s control• Investigation into the use of students’ prior

learning (RPL) as a form of assessment• Development of rubrics for all modules• Investigation into the use of Ipsative

assessment as a part of formative assessment (Hughes, Okumoto & Crawford, 2010)

Bibliography• American Psychology Association (APA). 2002. Principles of good practice in distance education and their application to

professional education and training in psychology. Washington, DC: APA. Available www.apa.org/ed/resources/finalreport.doc

• Brenner, E. & Shalem, Y. 2010. Immediate response to mediated learning: The use of technology for continuous assessment in higher education: CHE (2010) Teaching and Learning beyond formal access: Assessment through the looking glass: Higher Education Monitor No. 10 of August 2010.

• Commonwealth of Learning (COL). 2004. Planning and Implementing Open and Distance Learning Systems: A Handbook for Decision Makers. Vancouver: COL downloaded from www.col.org .

• Council on Higher Education (CHE). 2004. Criteria for Institutional Audits, Higher Education Quality Committee, June 2004. Pretoria: CHE.

• Council on Higher Education (CHE). 2004. Criteria for Programme Accreditation, Higher Education Quality Committee, November 2004. Pretoria: CHE.

• Hendrikz, J. & Aluko, FR. 2011. Quality assurance in ODL & its possible impact on students’ attrition rates. Paper presented at the 3rd ACDE conference, 12-15 July, Dar-es-Salaam, Tanzania.

• Hughes, G., Okumoto, K. & Crawford, M. 2010. Ipsative assessment and motivation of distance learners. London: University of London, Centre for Distance Education.

• Shavelson, RJ. (n.d.) On the Integration of Formative Assessment in Teaching and Learning with Implications for Teacher Education.

• University of Pretoria. 2009. Distance Education policy. Pretoria: UP.• University of Pretoria. 2010. Data from the Unit for Distance Education. Pretoria: UP.• Welch, T. & Reed, Y. Eds. 2005. Designing and Delivering Distance Education: Quality Criteria

and Case Studies from South Africa. Johannesburg: Nadeosa.

THANK YOU

ruth.aluko@up.ac.zaephraimm@saide.org.za