Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Purpose of Assessment and Feedback Combined approach to improve
teaching and learning whilst informing reporting and accountability
Lenore AdieAssociate Professor in Teacher Education and Assessment
Rowntree, 1987
How shall we know them?
Assessment Purpose
FormativeProgressing learning during teaching
Next-step learningContent + Process Learning goals
FeedbackRange of variables
Dialogic
SummativeReporting
Next-step learningContent + Process Learning goals
Feedback
Reporting
Range of variables
Why are you assessing?
How do you want them to know it?
Exactly what knowledge do
you want students to
have?
What task(s) will the students perform to
communicate their knowledge?
How will you analyse and interpret evidence?
What will you accept as evidence that a
student has the desired knowledge?
Assessment purpose - Assessment design
Adapted from Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016)
Educational assessment is at heart an exercise in evidentiary reasoning. (Mislevy & Riconscente, 2005, p. iv)
design of formative
assessment opportunities
high quality, multiple data
sources
attend to descriptive features of
student work
connect the data with instructional
plans -differentiation of
instruction
Appraise the information for validity
and relevance
formative feedback is a
finer grain size
Evidentiary decision-making
The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it.
Sadler (1989. p. 121)
Transparency
Fairness
Fine-grained details
Engendering trust Motivation
Right to privacy
Sadler (1985)
3 difficulties in defining criteria
Lack of distinct boundaries
Role of experience on interpretation
Unanticipated qualities
How to enable students to be better learners through knowledge of
criteria and developing expertise in their use?
reduce trial and error attempts in students’ efforts to produce ‘good work’
promote practices and provide artefacts that develop evaluative experience and expertise
4 enabling conditions
for including
students in assessment
practices
Teachers’ assessment identities
Students’ assessment identities
Social moderation
The role of artefacts
Teacher assessment identityKnowledge and skills
Beliefs, confidence, role, feelings
Development of expertiseProfessional standards
Evidentiary decision makingUse of data and evidence; artefacts.
Moderation Looney,Cumming,vanderKleij &Harris(2017)
(Adie, 2016)
Gra
du
ate
(G
)P
roficie
nt
(P)
Hig
hly
A
ccom
plish
ed
(HA
)
Lea
d
(L)
5.2 Provide Feedback to Students on Learning S
tud
en
tsS
cho
ol
PurposeContentAction
Effective Ran
gePurpose
Info
rmed
, Tim
ely
Judg
emen
ts
ModelL:
InitiateL:
About Current Needs in order to Progress Learning
HA:SelectHA:
ProvideP:
Demonstrate an Understanding
G:
ProgramsL:
Exemplary PracticeL: Support ColleaguesL:
About Achievement Relative to Learning Goals
P:
About LearningG:
+ Targeted+ EffectiveTimely, Appropriate Feedback
HA:P:G:
(Adie, Stobart & Cumming, forthcoming)
ArtefactsExemplar Cognitive commentary
Overall this student’s work sample is best matched with a ‘Developing +’ standard. This student generally expressed a point of view, used the exposition framework, provided some supporting evidence for the viewpoint and included some relevant details to provide information about life in the Gold Fields which are matched with the ‘Consolidating’ standard. However, the student needed to have written a final statement that referred to the viewpoint, explained the reason for her point of view in the thesis and made use of more specific conjunctions to strengthen the argument to achieve an overall ‘Consolidating’ standard. Because this student did not offer a conclusion but demonstrated a ‘Consolidating’ standard against 4 criteria, on balance, this work has been rated at no more or no less than a Developing +’ standard.
From the M eeting in the M iddle project (W yatt-Sm ith & B ridges, 2007, p . 8 )
Adie & W illis (2016).
Moderation insights
… you never really understand what they’re [criteria and standards] about until you are grading or you are using them. So, until you see them in operation it’s hard to know, but there is a danger of being too detailed and almost verbose with what you’re trying to do.
(Klenowski & Wyatt-Smith, 2010, p. 34)
Teacher 1: …what I have found is that it is impossible to design criteria that work all the time and what you need to do is that you need to use them more as a …
Teacher 2: …a guideline?
Teacher 1: …a rough guideline and then you need to see.
(Wyatt-Smith, Klenowski, & Gunn, 2010, p. 68)
Student assessment identityKnowledge and skills
Beliefs, confidence, role, feelings
Development of expertiseSelf-assessment; Peer assessment
Evidentiary decision makingUse of data and evidence; artefacts.
Feedback as a dialogic process
Van der Kleij, Adie, Cumming, 2016
Artefacts Criteria
Self-assessment
Panadero, Alonso-Tapia, and Huertas, 2012;
Fletcher, 2016
Chang, Liang, & Chen, 2013;
Wyatt-Smith & Bridges, 2007
higher-performing students may be more inaccurate in SA
(Hosein & Harle 2018)
A
Define the criteria by
which students assess
their work
B
Teach students how
to apply the criteria
CGive students
feedback on their
self-assessments
DGive students help in
using self-assessment
data to improve
performance
EProvide sufficient
time for revision
after self-
assessment
F
Do not turn self-
assessment into
self-evaluation by
counting it toward
a grade
Se
lf-asse
ssm
en
t: Imp
lica
tion
s fo
r p
rac
tice
Panadero, Jonsson, & Strijbos, 2016, p. 306
Peer assessment
Guiding questions, prompts Rotsaert, Panadero, Schellens, & Raes(in press); Gan & Hattie (2014)
Rubrics, Exemplars -Knowledge of standards and criteriaGan & Hattie (2014); Rotsaert, Panadero, Schellens, & Raes (in press)
Co-constructing and deconstructing criteriaLeenknecht & Prins (in press)
Psychologically safe conditions — TrustHarris & Brown (2013); Panadero, (2016)
Guidance and instructionTsivitanidou, Zacharia & Hovardas, (2011); Hovardas et al. (2014)
Practice
(Adie, Stobart & Cumming, forthcoming)
Site validity
Localisedknowledge,
situated contexts
System validity
Political accountability / improvement agendas, generalisable across sites
(Freebody & Wyatt-Smith, 2004; Adie & Wyatt-Smith, 2018 in-press)
Rowntree, 1987
How shall we know them?
Theoretical frameworkRobinson & Taylor (2007, p.8) four core values of student voice
A conception of communication as
dialogue
Dialogue enables
development of shared
understanding
The requirement for participation and democratic
inclusivity
Participation and democratic
inclusivity require all
voices to be listened to; and acceptance of
diversity
The recognition that power
relations are unequal and problematic
Equal power means
equitable opportunities
to actively express ideas and to shape
consequences
The possibility for change and
transformation
Transformation actively seeks
(student) voice, takes it
seriously and uses it to
creatively solve problems
Theoretical ways forward -Student Voice Lundy (2007)
Space to express a view
Coaching to communicate their learning
Action on views expressed
Audience to hear views
Reconceptualising reporting
teacher ownership of
data reinterpretation throughdialogue
participation and
inclusiveness mutual
partnership
expertise of teacher,
student, and family
reporting
progressing learningassessment
communicating
Combined approach to improve teaching and learning whilst informing reporting and accountability
Dialogic feedback
System and site validity
Reporting: Student voice
Student self- and peer assessment
How shall we know them?
Assessment designStandards-referenced assessment as an interconnected system
Assessment as shared inquiry
Assessment identity
ReferencesAdie, L. (2016). Working in a system of standards-referenced assessment: Traversing the intersections. In H. Fehring (Ed.). Assessment into Practice. Primary English Teaching Association Australia (PETAA).
Adie, L. & Willis, J. (2016). Making meaning of assessment policy through teacher assessment conversations. In D. Laveault and L. Allal (Eds.). Assessment for Learning: Overcoming the Challenges of Implementation. (pp. 35-53). Dordrecht, the Netherlands: Springer.
Adie, L., & Wyatt-Smith, C. (in-press 2018). What is the potential of standards, validation and moderation for demonstrating quality in initial teacher education? In C. Wyatt-Smith & L. Adie (Eds.), Teacher education, learning innovation and accountability. Springer.
Chang, C.-C., Liang, C., & Chen, Y.-H. (2013). Is learner self-assessment reliable and valid in a web-based portfolio environment for high school students? Computers & Education, 60(1), 325–334.
Fletcher, A. K. (2016). Exceeding expectations: Scaffolding agentic engagement through assessment as learning. Educational Research, 58(4), 400–419.
Freebody, P., & Wyatt-Smith, C. (2004). The assessment of literacy: Working the zone between system and site validity. Journal of Educational Enquiry, 5(2), 30-49.Gan, M. S., & Hattie, J. (2014). Prompting secondary students’ use of criteria, feedback specificity and feedback levels during an investigative task. Instructional Science, 42(6), 861–878.
Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation. Teaching and Teacher Education, 36(0), 101-111.
Hovardas, T., Tsivitanidou, O. E., & Zacharia, Z. C. (2014). Peer versus expert feedback: An investigation of the quality of peer feedback among secondary school students. Computers & Education, 71(Supplement C), 133–152.
Klenowski, V. & Wyatt-Smith, C. (2010). Standards, teacher judgement and moderation in contexts of national curriculum and assessment reform. Assessment Matters, 2, 107-131.
Leenknecht, M. J. M., & Prins, F. J. (in press). Formative peer assessment in primary school: The effects of involving pupils in setting assessment criteria on their appraisal and feedback style. European Journal of Psychology of Education, 1–16. doi:10.1007/s10212-017-0340-2
Looney, A., Cumming, J., Van Der Kleij, F., & Harris, K. (2017). Reconceptualising the role of teachers as assessors: Teacher assessment identity. Assessment in Education: Principles, Policy & Practice, 1–26. https://doi.org/10.1080/0969594X.2016.1268090
Lundy, L. (2007). ‘Voice’ is not enough: conceptualising Article 12 of the United Nations Convention on the Rights of the Child. British Educational Research Journal, 33(6), 927–942. Mislevy, R. J., & Riconscente, M. M. (2005). Evidence-centered assessment design: Layers, structures, and terminology. Menlo Park, CA: SRI International.
Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 247–266). New York, NY: Routledge.
Panadero, E., Alonso-Tapia, J., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813.
Panadero, E., Jonsson, A., & Strijbos, J.-W. (2016). Scaffolding self-regulated learning through self-assessment and peer assessment: Guidelines for classroom implementation. In D. Laveault & L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 311–326): Cham, Switzerland: Springer International.
Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016). A framework for conceptualizing and evaluating the validity of instructionally relevant assessments. Educational Psychologist, 51(1), 59-81.
Robinson, C., & Taylor, C. (2007). Theorizing student voice: values and perspectives. Improving Schools, 10(1), 5–17. Rotsaert, T., Panadero, E., Schellens, T., & Raes, A. (2017). “Now you know what you’re doing right and wrong!” Peer feedback quality in synchronous peer assessment in secondary education.
European Journal of Psychology of Education, 1–21. doi:10.1007/s10212-017-0329-xRowntree, D. (1987). Assessing students: How shall we know them? (Rev. ed.). London: New York: Kogan Page; Nichols Pub.
Sadler, D. R. (1985). The origins and functions of evaluative criteria. Educational Theory, 35(3), 285–297.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144.
Tsivitanidou, O. E., Zacharia, Z. C., & Hovardas, T. (2011). Investigating secondary school students’ unmediated peer assessment skills. Learning and Instruction, 21(4), 506–519. Van der Kleij, F., Adie, L., & Cumming, J. (2017). Using video technology to enable student voice in assessment feedback. British Journal of Educational Technology, 48(5), 1092-1105.
Wyatt-Smith, C. M., & Bridges, S. (2007). Meeting in the middle – assessment, pedagogy, learning and educational disadvantage. Evaluation Study for the Department of Education, Science and Training on Literacy and Numeracy in the Middle years of Schooling Initiative Strand A, Queensland Project Report.
Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgment practice in assessment: A study of standards in moderation. Assessment in Education: Principles, Policy & Practice, 17(1), 59–75.
Associate Professor Lenore Adie