Upload
jonas-jacobs
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Paul Kleiman PALATINE Lancaster University
Chaos and Crystals: negotiating assessment
orAssessing at the edge of chaos
Chaos and Complexity
Performance Theory
Personal Experience
Research Paradigms
Assessment
Chaos and Complexity
Performance Theory
Personal Experience
Research Paradigms
Assessment
SOME STARTING POINTS
Performer/Teacher
Text/Curriculum
Audience/ Students
Performance/ Learning Space
Assessment
c h a o s a n d c o m p l e x i t y
Self-organisation
Paradox
Emergence
The edge of
chaos
operating on the edge of chaos
STASIS
Close to certainty
Close to agreement
Far from agreement
Far from certainty(Based on Stacey 2000)
zone of complexity
CHAOSEdge of Chaos
– tension AND
creativity
Simple
Complicated
Complex
Students
Failure
Finance
Staff
RealityHEI
AdministrationResits Paperwork
generator
Learning &Teaching
Understanding
Research
ExternalExaminer
GraduatesStress
Knowledge
Skills
Students
Failure
Finance
Staff
RealityHEI
AdministrationResits Paperwork
generator
Learning &Teaching
Understanding
Research
ExternalExaminer
GraduatesStress
Knowledge
Skills
Closed system
Knowledge as external and transmitted
Predictability
Linear causality
Deterministic
the problem• The more assessment involves qualitative information, the
more subjectivity is involved• Stricter assessment criteria + more structured and proscribed
content = improved reliability• BUT…..• above would “obliterate the essence of qualitative assessment
in terms of flexibility, personal orientation and authenticity”.
Driessen et al (2005) Medical Education 39: 214–220
•Positivist
AimExplanation, control
KnowledgeObjective, measurable, value-free, universal, generalisable, external, quantifiable, can be transmitted and
acquired
Teacher/Researcher Expert, independent
CriteriaRigour via ‘holy trinity’ of validity, reliability and generalisability
There’s a reality ‘out there’ that can be studied, captured and understood
•Interpretive
AimUnderstanding
KnowledgeSubjective, contextualised, value-dependent, multiple-
realities
Teacher/Researcher Independent participant
Criteria‘Rigour’ (?) via credibility, transferability, dependability,confirmability
‘Truth’ is a matter of consensus amongst informed and sophisticated constructors, not of correspondence with an objective reality.
• Interpretive
‘Facts’ have no meaning except within some value framework; hence there cannot be an ‘objective’ assessment of any proposition.
Phenomena can be understood only within the context in which they are studied; findings from one context cannot be be generalised to another; neither problems nor their solutions can be generalised from one setting to another.
Note: the problem with ‘evidence-based practice’ , ‘best practice’, ‘transferable skills’, ‘benchmarks’, ‘national standards’, etc.
Therefore: Find and replace
• Replace
• Validity with Credibility, Coherence,
Consistency, Trustworthiness, Authenticity
• Certainty with Relativity
• Generalised Explanation with Local
Understanding
• Source Data with Empirical Materials
(which can become data…. and then
evidence)
• Is it true? with Does it work?
• Single Point Perspective with Multiple
Perspective
•The Triangle with the Crystal
The complex and
multidimensional nature of
learning cannot be captured
effectively and
comprehensively by any
single instrument or
analytical procedure.
The complex and
multidimensional nature of
learning cannot be captured
effectively and
comprehensively by any
single instrument or
analytical procedure.
The use of multiple methods
reflects an attempt to secure an
in-depth understanding of the
phenomenon in question.
However…..
from TRIANGULATION to CRYSTALLISATION
Triangulation carries with it the
image of a mathematical
procedure adding rigour and
discipline in one sense, but in
another sense restricting the
research to one of scientific
method and a positivist framework
where variables are few and can
be controlled or manipulated.
(Chien, 2004)
A
BC
P
Ad sedere / to sit down
together
"You've got to involve students
actively, not just view them as
objects of assessment, but as
agents of assessment. This can
be done in many ways. One is
that you ask students
systematically what they have
learned. It's a simple idea; it's
rarely done.....You find that
students say some remarkable
things.”
Walt Haney, Professor of Education,
Center for the Study of Testing, Evaluation and Educational
Policy, Boston, USA.
negotiating assessment
negotiating assessment
Presentation and Performance: Negotiated assessment
• Students engaged in creative practice work at different levels AND different ways.
• The products they create will be different, as will the processes and methods utilised, and the disciplines they represent
• That assessment should operate and be perceived as an integral part of the learning process rather than 'bolted-on' to the end of that process.
• That the form, content and implementation of the assessment process should be commensurable with the discourse and practices of the field
negotiating assessment
Six assessment fields:
• Presentation/Production i.e. the finished product presented to an audience
• Process i.e. the journey that led to the product
• Idea i.e. the ideas that informed both the process and the product.
• Technical i.e. the quality and utility of the technical features of the product and the skills with which they were assembled and/or operated
• Documentation i.e. research, design, sketches, planning, evaluation, analyses, portfolio, etc.
• Interview i.e. the student's ability to articulate their understanding, utilisation and application and use of any of the above.
negotiating assessment
• Learning contract – negotiated
• Regular meetings/tutorials
• Assessors see the performance/presentation + student compiles ‘portfolio of evidence’
• Assessment tutorial c. 30 min; individual / group + at least 2 assessors
• Work through the criteria - moving upwards
• Reach a point of maximum information, optimal achievement
• Agree a grade band
• Sense of ‘completeness’, ‘accomplishment
• A learning experience for all concerned
negotiating assessment