EmergentEvaluationPhil Wood and Palitha Edirisingha

An initial exploration of a formative framework for evaluating distance learning modules

Guiding Considerations:• Many students don’t fill in distance learning evaluations• Evaluations often too generalised• Evaluations are inherently retrospective• Evaluations often focus on activities, tutoring, resources, environments,

but rarely learning and student action. • Analysis often gets reduced to simplistic reflections on numbers

Are there alternatives?

• Ellery (2006) – multidimensional evaluation on a campus-based course (evaluation throughout the module, with both student and lecturer perspectives)

• Benson et al (2009) - Participatory evaluation, again multi-modal. Based on Jackson and Kassam 1998

a process of self-assessment, collective knowledge production, and cooperative actionin which the stakeholders in a development intervention participate substantively inthe identification of the evaluation issues, the design of the evaluation, the collectionand analysis of the data, and the action taken as a result of the evaluation findings.(1998, 3)

We wanted to develop an approach which allowed for:• Formative module evaluation• Linked to curriculum development• To make distance learning review more than a ‘performative’ activity• Putting pedagogy (interpenetration of teaching, learning, curriculum

and assessment and their interaction with teachers and students) at the centre of the process

• Emergence and trialling of new approaches as a standard element of our work

Basic outline of process

Possible Advantages• Allows us to develop elements of the curriculum in real time, with

student response and reflection helping shape the content and approach

• Students value the opportunity to give deeper, more critical views

• Deeper understanding of the complexities of pedagogies, with opportunity to respond and use formatively

• Rolling programme of curriculum renewal which is holistic – measured innovation and emergent understanding of student experiences/learning

• Synergy with research opportunities

• Opportunity to develop a model for campus-based use (perhaps reflecting Jackson and Kassam 1998)

Issues on Which to Reflect• Time

• Workload at points of development – intensive process

• Making time for ‘deep work’ (Newport, 2016) to draw out insights/reflections and emerging models of pedagogy.

• How does the data gained fit into increasingly simplistic and performative evaluation frameworks at university level? Does it matter?

References

Jackson, E.T., and Y. Kassam. 1998. Knowledge shared: Participatory evaluation in developmentcooperation. West Harford, CT: Kumarian Press

Newport, C. (2016) Deep Work, Rules for focused success in a distracted world. London, Little Brown.

Benson, R.; Samarawickrema, G. & O’Connell, M. (2009) ‘Participatory evaluation: implications for improving electronic learning and teaching approaches.’ Assessment & Evaluation in Higher Education, 34:6, 709-720.

Ellery, K. (2006) ‘Multi‐dimensional evaluation for module improvement: a mathematics‐based case study.’ Assessment & Evaluation in Higher Education, 31:1, 135-149.