Valuing evaluation: A Case Study of Professional Development to

Support Academic Engagement in Online Evaluation Processes and

Outcomes

Dr. Diana QuinnSenior Lecturer: Professional Development

University of South Australia

University of South Australia 2

Academic centred professional development

• Conceptual shift from ‘I will tell you this and therefore you will learn’

to:

‘I want to help you in ways which are effective for you and match your needs’.

• Starts planning professional development based on the current conceptions of academics

• Aims to meet academics’ needs (information and psychological)

University of South Australia 3

Case study: Understanding the learners

• Education Arts and Social Science Division at University of South Australia– 2003 academics : 238 FT; 66 fractional (planning);

260 sessional (email distribution list)– From 2005, evaluation of courses and teaching will

only be able to be administered online

• Academics current conceptions– Negative values about evaluation– Evaluation linked to promotion and teaching awards– Online evaluation has low response rate– Students have ‘evaluation overload’

University of South Australia 4

ACPD approach

• Strategies to improve response rates to online evaluation selected as topic

• Process– Identify those academics performing well– Interview, collate and share local best practice

• Strategies to improve accessibility to online• Strategies to improve students’ understanding of

evaluation

– Collaborative document produced– Model desired behaviour (model emails)

University of South Australia 5

Three indicators of change

1. Number of student responses to course evaluation

2. Academics responses to online survey about ACPD

3. Informal comments

University of South Australia 6

Number of responses to course evaluation in EASS

0

1000

2000

3000

4000

5000

6000

7000

number if CEIspublished inEASS

number ofresponsesreceived inEASS

Responses to online surveyItem Number of respondents

(n=15)

Agree Neutral Disagree

I thought that collating and sharing successful strategies from EASS academics to improve response rates was a useful way to conduct professional development

13 1 1

I trusted these strategies as I knew they had worked for my peers 8 6 0

I appreciated being sent the model emails suggesting how I could communicate with students about the CEI

13 1 0

The model emails were inappropriate for use with my students 1 5 9

I think I can continue to maximise response rates by applying these strategies

9 6 0

Compared to last year I have improved my understanding of online evaluation

9 6 0

Compared to last year, I feel more confident about using online techniques for evaluation

7 6 2

University of South Australia 8

Informal comments

I can’t thank you enough for this – it really helped the penny drop for me as far as

online evaluation was concerned (personal communication, 2003).

University of South Australia 9

Discussion

• Break the cycle of educators (and students) not valuing evaluation processes and outcomes

• Information needed to connected to academic’s current conceptions

• Collaboratively produced document was a comprehensive resource – trust the learner

• Academics psychological needs considered (belonging, power, freedom and fun)

University of South Australia 10

Conclusion• As long as there is no requirement for some type

of educational qualification for academics to teach at university, there will be a need to provide meaningful and successful professional development about evaluation

• To make PD meaningful and successful, it needs to be academic-centred

• In ACPD, the information presented to academics needs to– recognise their current conceptions – respond to psychological needs

University of South Australia 11

University of South Australia 12

Questions

• What are the psychological needs of academics and how did they map against the professional development literature?

University of South Australia 13

Support the need to belong

• Sharing best practice (Rogers, 1983)• Foster peer learning in local environment (Boud

1999)• Develop local communities of practice (Ryan

2000)• Develop trust between academic and PD staff

(Ryan 2000)• Improve communication at all levels (Walsh &

Metcalf 2003)• Share local best practice, foster peer learning,

develop trust, use honest and open communication (Glasser 1999)

University of South Australia 14

Support the need for power• Let academics lead, own and participate in all

stages of the process (Walsh 2003)• Emphasise how it will improve the institution

(Walsh 2003)• Provide access to models and support to

complete task (Rogers 1983)• Make exemplars and scholarly practice visible

(Boud 1999, Ryan 2000, Walsh 2003)• Integration of local and centralised support

systems (Ryan 2000)• Involve learner in decision making process,

model what needs to be done, demonstrate the significance of the outcomes, provide adequate materials and support (Glasser 1999)

University of South Australia 15

Support the need for freedom

• Offer the capacity to try new processes without commitment (Rogers 1983)

• Build academic’s self esteem (Boud 1999)

• Allow exploration without commitment, provide models and examples for them to either develop their interpretation from, or choose to ignore (Glasser 1999)

University of South Australia 16

Support the need for fun

• Fun and humour are different

• Fun and learning interrelated

‘… when any of us is in any situation where we decide that we no longer want to learn, we stop having fun’ (Glasser 1988).

• Not a part of this project, but it is a consideration for future ACPD activities

University of South Australia 17

University of South Australia 18

Questions

• What is the psychological model that use used to plan ACPD?

University of South Australia 19

Glasser’s Choice theory: filters

Perceived World

K n o w l e d g e F i l t e r

V a l u e s F i l t e r

The Real World

University of South Australia 20

Glasser’s Choice theory: perceived world

Perceived World

K n o w l e d g e F i l t e r

V a l u e s F i l t e r

The Real World

University of South Australia 21

Glasser’s Choice theory: Basic needs

Belonging

Power

Freedom

Fun

Perceived World

K n o w l e d g e F i l t e r

V a l u e s F i l t e r

The Real World

Needs

University of South Australia 22

Glasser’s Choice theory: Quality world

Belonging

Power

Freedom

Fun

Quality World

Perceived World

K n o w l e d g e F i l t e r

V a l u e s F i l t e r

The Real World

Needs

University of South Australia 23

Comparison

Behaviour Choices

Glasser’s Choice theory: Choose to change

Belonging

Power

Freedom

Fun

Quality World

Perceived World

K n o w l e d g e F i l t e r

V a l u e s F i l t e r

The Real World

Needs

University of South Australia 24

University of South Australia 25

Make sure you are heard

• Use language and context that is relevant to the learner (Learner-centred) (Various, including Moore and Patrick 2002)

• Ensure the proposed changes are compatible with academics current practice (Rogers 1983)

• Recognise the current knowledge and values held by learner and present information in a format that they will engage with (Glasser 1999)