Upload
jwaycott
View
394
Download
0
Embed Size (px)
DESCRIPTION
Slides from a presentation given by Kathleen Gray and Jenny Waycott for a Medical Education Unit seminar series at the University of Melbourne in June 2010. From the ALTC-funded project "Web 2.0 Authoring Tools in Higher Education: New Directions for Assessment and Academic Integrity".
Citation preview
Good practice in using Web 2.0 to assess student learning in higher education
An ALTC Priority Project 2009-2011
Kathleen Gray & Jenny WaycottHealth Informatics & Virtual Environments Research Group
University of Melbourne
Medical Education Unit, University of MelbourneSeminar 30 June 2010
Project team
Jenny Waycott (project manager), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne.
Celia Thompson, School of Languages and Linguistics, University of Melbourne.
Margaret Hamilton, School of Computer Science and IT, RMIT University.
Joan Richardson, School of Business Information Technology, RMIT University.
Kathleen Gray (project leader), Faculty of Medicine / Department of Information Systems, University of Melbourne.
Rosemary Clerehan, Faculty of Medicine, Nursing and Health Sciences, Monash University.
Judithe Sheard, Faculty of Information Technology, Monash University.
Project background
The future of scientific & scholarly communication
Chodorow
(2000, p.91)
Chodorow, S. (2000). Scholarship & scholarly communication in the electronic age. Educause Review, 35(1), 86-92. http://www.educause.edu/ir/library/pdf/ERM001B.pdf
• “the form and substance of scholarly communications will change over time, so that it will be difficult to trace the historical flow of the work”
• “a free-flowing stream of scholarly discourse will reduce the role of scholarly authority in the progress of research”
• “the roles of individual authors will be obscured in the electronic environment”
Use of Web 2.0 in university learning and teaching
‘Web 2.0 and emerging technologies in online learning’
‘Analysis of 10 popular Web 2.0 tools used in higher education’
‘Facilitating new forms of discourse for learning and teaching: harnessing the power of Web 2.0 practices ’
‘The changing space of research: Web 2.0 and the integration of research and writing environments’
‘Can Web 2.0 and social software help transform how we measure quality in teaching, learning, and research?’
Implications for university learning, teaching & assessment?
O’Reilly & Battelle
(2009, p. 2)
O’Reilly, T., & Battelle, J. (2009). Web Squared: Web 2.0 Five Years On. Special Report for the Web 2.0 Summit, 20-22 October , San Francisco CA. http://assets.en.oreilly.com/1/event/28/web2009_websquared-whitepaper.pdf
“One of the fundamental ideas underlying Web 2.0 [is] that successful network applications are systems for harnessing collective intelligence ... a large group of people can create a collective work whose value far exceeds that provided by any of the individual participants”
Implications for university learning, teaching & assessment?
Kakutani
(2010,
paras 13-14)
Kakutani, M. (2010, 17 March). Texts without context. [Book review]. New York Times. http://www.nytimes.com/2010/03/21/books/21mash.html?ref=books
“jump to the summary, the video clip, the sound bite — never mind if context and nuance are lost in the process; never mind if it’s our emotions, more than our sense of reason, that are engaged; never mind if statements haven’t been properly vetted and sourced”
“tweet and text one another during plays and movies, forming judgments before seeing the arc of the entire work”
“power-search for nuggets of information that might support their theses, saving them the time of wading through stacks of material that might prove marginal but that might have also prompted them to reconsider or refine their original thinking”
Project aims
Academic standards, academic integrity, assessment principles ...
How to apply these to assessing student web 2.0 activities?
1. Survey and interview teaching academics (September 2009)
2. Draft guidelines for good practice (November 2009)
3. Pilot with teachers and students (February - June 2010)
4. Produce and share resources (July 2010 ff)
Current Web 2.0 assessment practices
Some preliminary findings
about how, where and why
60 Australian academics
are using Web 2.0
for assessment of student learning
that is more than just formative
(i.e. that earns marks in a subject)
Current Web 2.0 assessment practices: How
Type of Web 2.0 activity Number of responses
Wiki writing 32
Blogging/microblogging 31
Social networking 17
Audio/video podcasting 16
Virtual world activities 12
Social bookmarking 11
Current Web 2.0 assessment practices: How
Where students complete assignment Number of
responses
Off campus elsewhere (e.g., at home during
independent study time)52
On campus but out of class 25
On campus in class 16
Off campus while undertaking fieldwork or
workplace learning7
Current Web 2.0 assessment practices: How
How much the assignment is
worth
Number of responses
01-10% 7
11-20% 11
21-30% 9
31-40% 6
41-50% 9
51-60% 2
61-70% 0
71-80% 3
81-90% 2
91-100% 4
Current Web 2.0 assessment practices: Where
Field of Study Number of respondents
Humanities / Society & Culture16
Education15
Information Technology11
Medicine & Health9
Management & Commerce6
Other 3
Current Web 2.0 assessment practices: Where
Number of students
enrolled in unit
Number of responses
Less than 50 21
50-100 10
101-200 9
More than 200 7
Current Web 2.0 assessment practices: Where
Level of study Number of responses
Bachelor or honours degree 35
Postgraduate coursework degree 16
Current Web 2.0 assessment practices: Why
Intended learning outcomes Number of
responses
Generic or graduate skills or attributes 35
Specialised knowledge or skills required in a
discipline or profession29
Foundation knowledge or skills preparatory to
a discipline or profession28
Medicine/Health Example 1
• Individual students:
– Develop a wiki page on an assigned topic of relevance to the unit of study.
– Publish it for peer review.
– Review another student's wiki page on another topic.
– Reflect on the feedback received on/edits made to their wiki page
• Graded on a five point scale (F, P-, P, P+). Grades are awarded for criteria in five capability areas
• Encourages deep engagement with the topic, involves peer feedback, develops materials for use by all students.
Medicine/Health Example 2
• Students interview working biomedical scientists, review each others’ reports and publish final marked reports on a type of social networking site.
• Worth 11-20% of their subject mark
• Published for: “High school students ... looking for careers in science don’t know the names of many disciplines, they certainly don’t know the names of any people ... it’s a bit like a dating site. The visitors answer the same questions and the system pulls up the people who match.”
Medicine/Health Example 3
• Students are allocated to groups and given a topic, on which they must create and upload an educational and informative “document“ and give a class presentation.
• Groups may choose to create a video clip or a podcast (or a poster or flyer or similar).
• The students’ “document” and talk are assessed by peers and course coordinator/ faculty member.
• Part of the subject learning experience is for students and the teacher to negotiate assessment criteria to be used.
What would good practice look like?
National roundtable
Three checklists:
Affordances of Web 2.0 tools and technologies
Assessment processes
Assessment and related academic policies
What would good practice look like? Affordances
Affordances checklist ...
What is an appropriate fit between what assessment is trying to achieve and what Web 2.0 can do?
• Open publishing
• Communication styles and texts
• Personal identity and experience
• Co-creation, collaboration, crowdsourcing
• Content management
What would good practice look like? Affordances
Open publishing:
• Student work can be made easily accessible to an audience of peers for mutual benefit including reviewing and rating.
• Review and assessment of student work from outside the university can be invited or anticipated.
What would good practice look like? Affordances
Communication styles & texts
• Web 2.0 assignments can involve frequent short pieces of work employing conversational language and combining audio, video, images & text.
• Feedback can be exchanged rapidly, using rating or ranking systems, informal rejoinders, audio, video, images, icons.
What would good practice look like? Affordances
Personal identity & experience: • Students’ online identity can be
different from the student who is recognisable in class.
• Students’ social or cultural experiences of web authoring can influence the work they produce for assessment.
• Reflection and self-reflection about the idea of identity are prompted by the need to create and express an online identity.
What would good practice look like? Affordances
Co-creation, collaboration, crowdsourcing:
• Group work can scale between a small closed group and a large free-to-join learning community
• Individual contributions to group work can (sometimes) be distinguished.
• Groups can work on large, complex tasks.
What would good practice look like? Affordances
Content management
• Students’ assessable work may consist of remixing web content from diverse sources.
• Students’ assessable work may be posted on several host sites. Work posted on one site may be syndicated by others and tracked back.
• Students can control the content they produce for assessment in accordance with terms of service, end user agreements or other governance policies of host sites.
What would good practice look like? Processes
Processes checklist ...
How do teachers use Web 2.0 to support student, self- and organisational learning throughout the cycle of activities involved in the assignment?
Design
Implement
MarkFeedback
Review
What staff have said about ...
Designing the assignment
Implementing the assignment
Marking the assignment
Giving results and feedback to students
Reviewing how well the assignment works
What would good practice look like? Policies
Policies checklist ...
How can assessment using Web 2.0 be made safe and fair for students and staff?
• disability
• access to IT services or equipment
• appropriate conduct
• identity and privacy
• academic honesty and integrity
• special consideration
• moral rights and copyright
Examples where policies are not clearly observed
Policy area % Not sure
Copies of students’ marked work are available if there is a need to deal with appeals/complaints
20
This assignment encourages academic honesty and integrity 20
Students’ identity and privacy in online environments are safeguarded
20
Students are provided with timely feedback on marked work for this assignment
20
This assignment provides for equitable assessment for students with a disability
23
Students’ moral right and copyright in work they produce are protected
27
Students whose work shows evidence of cheating or misconduct are formally disciplined
28
What can (and can’t) be done in real subject teaching settings?
Draft guidelines pilot-tested
for practicability
in 17 subjects
at 5 universities
in Victoria
during Sem 1, 2010
BloggingCinema Studies / Criminal Law
Cultural Studies / Media Studies
Social bookmarking Education
Social networking Languages
Video sharing Business / Economics
Photo sharing Communication Design
Virtual worlds Languages
Wiki writing
Accounting / Education
Information Technology
Languages / Science
Combined
Web 2.0 tools
Information Management
Information Technology
Sharing project progress
Moodle: www.groups.edna.edu.au/course/view.php?id=2146
Blog: http://web2assessment.blogspot.com
Bookmarks: www.citeulike.org/tag/assessment20
Webinar: www.transformingassessment.com/events_26_may_2010.php
Workshops 2010-11 @ HERDSA, ATN Assessment, ASCILITE, ACE
Papers: • Gray, K., Thompson, C., Clerehan, R., Sheard, J., & Hamilton, M. (2008). Web 2.0
authorship: Issues of referencing and citation for academic integrity. The Internet and Higher Education. 11(2), 112-118.
• Gray, K., Thompson, C., Sheard, J., Clerehan, R., & Hamilton, M. (2010). Students as web 2.0 authors: Implications for assessment design and conduct. Australasian Journal of Educational Technology, 26(1), 105-122.
Acknowledgements
Project Advisory Group• Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Buckworth,
Denise Chalmers, Geoffrey Crisp, Leitha Delves, Bobby Elliott, Jacqui Ewart, Glenn Finger, Tom Franklin, Merrilyn Goos, Scott Grant, Ashley Holmes, Christopher Hughes, David Jones, Marj Kibby, Adrian Kirkwood, Mark Lee, Catherine McLoughlin, Beverley Oliver, Kaz Ross, Alison Ruth, Royce Sadler, Mary Simpson, Arthur Winzenried, Katina Zammit, Lynette Zeeng.
Project Reference Group• Michael Abulencia, Robyn Benson, John Benwell, Marsha Berry, Marilys Guillemin, Laura
Harris, Deborah Jones, Gregor Kennedy, Shaun Khoo, George Kotsanas, Lauren O’Dwyer, Jason Patten, Emma Read, Julianne Reid, Gordon Sanson, Cristina Varsavsky.
Project Pilot-testing Group• Matthew Absolom, Anne Davies, Cathy Farrell, Scott Grant, Terry Hallahan, Michael
Henderson, John Hurst, Ramon Laboto, Warren McKeown, Michael Nott, Kerry Pantzopoulos, Michele Ruyters, Michael Smith, Sandra Smith, Robyn Spence-Brown, Elizabeth Stewart, John Terrell, Jenny Weight, Lynette Zeeng
ALTC Support for this project has been provided by the Australian Learning and Teaching Council Ltd. (www.altc.edu.au), an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this presentation do not necessarily reflect the views of the Australian Learning and Teaching Council, or the views of individual contributors apart from the project team.