The challenge of classroom classroom--based based assessment … Lyons The... · Ethical principles Classroom assessment must be fair. ... how well the assessment practices are interwoven

Liz HampLiz Hamp--LyonsLyons

The challenge of The challenge of classroomclassroom--based based

assessmentassessment

Copyright Liz HampCopyright Liz Hamp--Lyons 2013Lyons 2013 22

How formal highHow formal high--stakes tests relate to stakes tests relate to actual language learningactual language learning**

Do you agree that …… is a(n) effective way to learn English

useful TOEFL preparation activity

%* rank %* rank writing essays 100 1 91 4 giving oral presentations 100 1 24 17 listening to real-life English 100 1 82 7 reading real-life English texts 100 1 71 8 writing a summary of something you have read 100 1 50 11 writing a summary of something you have listened to 100 1 50 11 giving an oral summary of something you have read 100 1 45 13 giving an oral summary of something you have listened t 100 1 41 15 completing listening comprehension exercises 96 9 100 1 having conversations in English 96 9 41 15 completing reading comprehension exercises 91 11 100 1 doing group projects 87 12 24 17 studying an English language learning textbook 87 12 45 13 learning grammar rules and doing grammar exercises 86 14 86 6 doing timed TOEFL tests 50 15 91 4 studying test preparation materials 48 16 96 3 memorizing examples of good essays 41 17 64 9 memorizing vocabulary lists 30 18 64 9

•Percentage of positive responses (Agree and Strongly agree)

Table 12: Comparison of students’ beliefs about useful TOEFL preparation activities and effective language learning activities


What is the fundamental issue?What is the fundamental issue?

How teachers teach matters.How teachers teach matters.

How teachers integrate assessment into their How teachers integrate assessment into their teaching matters.teaching matters.

Black & Wiliam defined formative Black & Wiliam defined formative assessment as assessment as ““all those activities all those activities undertaken by teachers, and/or their undertaken by teachers, and/or their students, which provide information to be students, which provide information to be used as feedback to modify the teaching and used as feedback to modify the teaching and learning activities in which they are engagedlearning activities in which they are engaged”” (1998, pp.7(1998, pp.7--8)8)


What is the fundamental issue?What is the fundamental issue?

How teachers teach matters.How teachers teach matters.

How teachers integrate assessment into their How teachers integrate assessment into their teaching matters.teaching matters.

Black & Wiliam defined formative Black & Wiliam defined formative assessment as assessment as ““all those activitiesall those activities undertaken by teachers, and/or their undertaken by teachers, and/or their students, which students, which provide informationprovide information to be to be used as feedbackused as feedback to to modify the teaching and modify the teaching and learning activitieslearning activities in which they are engaged in which they are engaged (1998, pp.7(1998, pp.7--8)8)


More than one challengeMore than one challenge

Terminology/definitionTerminology/definition

Identifying principlesIdentifying principles

Relating principles to second/foreign Relating principles to second/foreign language learning and teachinglanguage learning and teaching

Classroom assessment literacyClassroom assessment literacy


TerminologyTerminology

In the last decadeIn the last decade--plus, a good deal of attention in plus, a good deal of attention in the field of educational assessment has been the field of educational assessment has been turned to various forms of assessment that are turned to various forms of assessment that are not not ‘‘testingtesting’’: :

formative assessmentformative assessment

assessment for learningassessment for learning

schoolschool--based assessmentbased assessment

classroomclassroom--based assessment based assessment

learninglearning--oriented assessmentoriented assessment

(and more) (and more)


Principles: Principles: Questions to be consideredQuestions to be considered

Is classroomIs classroom--based assessment necessarily based assessment necessarily formative? Are there conditions under which it formative? Are there conditions under which it may not be formative? may not be formative?

Is overt, explicit feedback a necessary element of Is overt, explicit feedback a necessary element of classroomclassroom--based assessment? based assessment?

Can assessment be Can assessment be ““for learningfor learning”” (as in Black & (as in Black & Wiliam 1998; Black et al 2003) if it is taking place Wiliam 1998; Black et al 2003) if it is taking place in the classroom but being reported upwards to in the classroom but being reported upwards to accountability systems? accountability systems?


Classroom PrinciplesClassroom Principles

Setting clear goalsSetting clear goals

Designing appropriate learning and Designing appropriate learning and assessment tasksassessment tasks

Communication of assessment criteria to Communication of assessment criteria to teachers and studentsteachers and students

Provision of highProvision of high--quality feedbackquality feedback

Conscious provision of opportunities for Conscious provision of opportunities for selfself-- and peerand peer--assessmentassessment


Technical principlesTechnical principles

Teacher assessment should be the product Teacher assessment should be the product of professional judgement involvingof professional judgement involving

critical interpretation of evidence from student critical interpretation of evidence from student performances performances

multiple sources of evidencemultiple sources of evidence

application of knowledge of appropriate application of knowledge of appropriate measurement principlesmeasurement principles

value judgements about the meanings of value judgements about the meanings of resultsresults

Decision consistencyDecision consistency


Ethical principlesEthical principles

Classroom assessment must be fair.Classroom assessment must be fair.

It must be aware of its unintended as well It must be aware of its unintended as well intended consequences.intended consequences.

It should positively influence studentsIt should positively influence students’’ motivation and learning.motivation and learning.

It should be efficient and feasible for It should be efficient and feasible for teachers.teachers.

It should be empowering for teachers.It should be empowering for teachers.


A new kind of fairnessA new kind of fairness

In the US, Linda DarlingIn the US, Linda Darling--Hammond (2010) Hammond (2010) wrote an important paper in which she wrote an important paper in which she argued that while standardized testing is not argued that while standardized testing is not equitable, alternative assessment may not be equitable, alternative assessment may not be equitable either. She argues that all kinds of equitable either. She argues that all kinds of assessment assessment ““depend not only on the design depend not only on the design of the assessments themselves, but also on of the assessments themselves, but also on how well the assessment practices are how well the assessment practices are interwoven with the goals of authentic school interwoven with the goals of authentic school reform and effective teachingreform and effective teaching““ for their for their fairness. (p.5) fairness. (p.5)


Underlying processes or data Underlying processes or data collection methods that can inform collection methods that can inform

assessment in the classroomassessment in the classroom

ObservationObservation

Recorded lessonsRecorded lessons

Portfolio collectionPortfolio collection

Journals, questionnaires Journals, questionnaires

InterviewsInterviews

Recall protocolsRecall protocols

PeerPeer--peer talk recordingspeer talk recordings


A different way of seeingA different way of seeing

We can see that all of these are tools to We can see that all of these are tools to create a create a ‘‘windowwindow’’ through which we can through which we can see into the student see into the student beyond the product beyond the product of their learningof their learning

For classroom assessment, we want to For classroom assessment, we want to know not just what happened, but how and know not just what happened, but how and why it happened.why it happened.

We also want to know that soon enough to We also want to know that soon enough to put the knowledge to useput the knowledge to use..


Principles for Principles for languagelanguage classrooms?classrooms?

The third challenge in the context of a group The third challenge in the context of a group such as EALTA is to consider whether (and such as EALTA is to consider whether (and perhaps which of) the principles hold true perhaps which of) the principles hold true when working with second, third or fourth when working with second, third or fourth language learners, and also whether there language learners, and also whether there may be some additional principles that apply may be some additional principles that apply specifically to the teaching and learning of specifically to the teaching and learning of languages. languages.


Is the medium the message?Is the medium the message?

The temptation will always be to operationalize a The temptation will always be to operationalize a classroomclassroom--based language based language ‘‘assessmentassessment’’ as a as a series of miniseries of mini--tests of specific linguistic tests of specific linguistic knowledge knowledge –– instead we must always look to the instead we must always look to the purposepurpose of the assessment. of the assessment.

But we must also look to the underlying But we must also look to the underlying message about language that is projected by the message about language that is projected by the types of assessments we use. Some minitypes of assessments we use. Some mini--tests tests of discrete language areas serve a useful of discrete language areas serve a useful purpose; but need to be balanced by purpose; but need to be balanced by assessments that focus on the realassessments that focus on the real--life value of life value of language. language.


ReaRea--Dickins (2007): Dickins (2007): ““the means by which to the means by which to achieve consistency in making judgments about achieve consistency in making judgments about language samples are well rehearsed, but much language samples are well rehearsed, but much uncharted territory remains in the development uncharted territory remains in the development of quality formative assessment.of quality formative assessment.””

The types of criteria that become important in The types of criteria that become important in classroom language assessment must include classroom language assessment must include ““the provision of a rich array of opportunities for the provision of a rich array of opportunities for learners to use and stretch their linguistic learners to use and stretch their linguistic resourcesresources”” (p.516)(p.516)


In the context of language In the context of language assessmentassessment

HampHamp--Lyons 2007, Figure 1: two extremes of Lyons 2007, Figure 1: two extremes of assessment culturesassessment cultures

Classroom-based assessment Classical testing Fluency-focused Accuracy-focused Individual-focused Group- or ‘norm’-focused Achievement/progress-focused Proficiency-focused Learner-focused Language-focused Process-focused Product-focused Teachers’/students’ voices Rule-makers’ voices Leads to assessment of learning Leads to ‘teaching to the test’


Message about language or about Message about language or about learning?learning?

Black & Wiliam (1998) say: Black & Wiliam (1998) say: ““the core of the the core of the activity of formative assessment lies in the activity of formative assessment lies in the sequence of two actions. The first is the sequence of two actions. The first is the perception by the student of a gap perception by the student of a gap between a desired goal and his/her own between a desired goal and his/her own state (of knowledge, and/or understanding, state (of knowledge, and/or understanding, and/or skills). The second is the action and/or skills). The second is the action taken by the learner to close the gap in taken by the learner to close the gap in order to attain the desired goal.order to attain the desired goal.”” (p.21)(p.21)

This puts the student at the centre of their This puts the student at the centre of their own assessment.own assessment.


Students as assessorsStudents as assessors

It is becoming clear that progress on effective It is becoming clear that progress on effective classroomclassroom--based assessment will not move far based assessment will not move far forward until a body of practical support is forward until a body of practical support is developed to help teachers learn how to teach developed to help teachers learn how to teach their student to assess their own language their student to assess their own language development.development.

Among the very many studies of the value of Among the very many studies of the value of student feedback, most fail to convince because student feedback, most fail to convince because the support given to students was inadequate.the support given to students was inadequate.

We take it for granted these days that teachers We take it for granted these days that teachers need to learn about assessmentneed to learn about assessment–– the same is true the same is true for students.for students.


The fourth challengeThe fourth challenge……

…… is perhaps the greatest.is perhaps the greatest.

Since classroomSince classroom--based assessment is based assessment is both new and different, and yet at the both new and different, and yet at the heart of what good teachers already do, heart of what good teachers already do, how can teachers learn and practice the how can teachers learn and practice the skills of assessment in their classrooms? skills of assessment in their classrooms?

FurthermoreFurthermore——how can they learn and how can they learn and practice the skills of practice the skills of languagelanguage assessment assessment using formative principles?using formative principles?


Classroom assessmentClassroom assessment——simple (?) simple (?) tools for teacherstools for teachers

QuestioningQuestioning

Probing questions (Probing questions (‘‘stretchstretch’’ questions)questions)

Switchback questioning (drop/raise level)Switchback questioning (drop/raise level)

Collaborative questioningCollaborative questioning

FeedbackFeedback

CommentComment--only only ‘‘markingmarking’’

Peer feedbackPeer feedback

Feedback as feedFeedback as feed--forwardforward

Wait timeWait time


Teacher strategies that require Teacher strategies that require advanced understanding of how advanced understanding of how

language workslanguage works

Teachers sharing criteria with learners Teachers sharing criteria with learners

Setting targetsSetting targets

Deconstructing criteriaDeconstructing criteria

Group use of criteriaGroup use of criteria

Identifying progressIdentifying progress

SelfSelf--assessmentassessment

PeerPeer--toto--peer commentary and revisionpeer commentary and revision


BUTBUT----““an activity or elicitation procedure in itself an activity or elicitation procedure in itself is neutral. It is only in its implementation is neutral. It is only in its implementation and the use to which the data that emerges and the use to which the data that emerges from a given activity is put that there from a given activity is put that there develops its formative or summative develops its formative or summative potentialpotential””““teachers may be trapped in their thinking teachers may be trapped in their thinking about assessment between two potentially about assessment between two potentially competing paradigms: to measure competing paradigms: to measure achievement and progress on the one achievement and progress on the one hand, or to facilitate through different hand, or to facilitate through different means their studentsmeans their students’’ language learning on language learning on the other.the other.”” (Rea(Rea--Dickins 2011, p. 514)Dickins 2011, p. 514)


The dual roles of teachersThe dual roles of teachers

This the main site of tension in assessmentThis the main site of tension in assessment

In the Hong Kong SBA we not only In the Hong Kong SBA we not only developed as humanistic and interactive an developed as humanistic and interactive an assessment of speaking as we could assessment of speaking as we could conceive within our parameters, we conceive within our parameters, we provided a large amount of teacher PD, provided a large amount of teacher PD, and onand on--site support, and we did this over a site support, and we did this over a 55--year period.year period.


BUT still longitudinal research and PhDs BUT still longitudinal research and PhDs by our students show that some teachers by our students show that some teachers have interpreted SBA in ways different have interpreted SBA in ways different from each other and in some cases far from each other and in some cases far from the original intention and philosophy, from the original intention and philosophy, but also sometimes far from the but also sometimes far from the specifications and guidelines provided in specifications and guidelines provided in detail.detail.


Understanding the tensionUnderstanding the tension

Black & Wiliam (1998) argue that it is not Black & Wiliam (1998) argue that it is not possible to introduce or strengthen possible to introduce or strengthen formative assessment in classrooms formative assessment in classrooms without some radical change in overall without some radical change in overall pedagogy because formative assessment pedagogy because formative assessment demands a demands a ‘‘constructivist classroomconstructivist classroom’’..

A classroom in which formative A classroom in which formative assessment is the norm and is effective assessment is the norm and is effective requires changes in the teacher and in the requires changes in the teacher and in the students as well as in the assessment students as well as in the assessment methods.methods.


Rising to the challengeRising to the challenge

Muchun Yin (2010) strikes a positive note Muchun Yin (2010) strikes a positive note when he talks about the spread of teacher when he talks about the spread of teacher thinking research into research on thinking research into research on assessment.assessment.

We are increasingly coming to the We are increasingly coming to the understanding that, if we are to change understanding that, if we are to change assessment practice, we must impact assessment practice, we must impact teachersteachers’’ core beliefs and understanding core beliefs and understanding about what testing/assessment is for, and about what testing/assessment is for, and how they might affect practice.how they might affect practice.


Teacher assessment literacyTeacher assessment literacy

Professional development, training materials for Professional development, training materials for specific classroom assessment methods, specific classroom assessment methods, guidelines for giving feedback guidelines for giving feedback –– these and many these and many other resources are needed BUT first teachers other resources are needed BUT first teachers need to be need to be ‘‘assessment literateassessment literate’’..

Good language teachers are already and always Good language teachers are already and always looking, listening, monitoring looking, listening, monitoring –– they are they are collecting data about their students collecting data about their students -- all their all their students.students.

The problem for many teachers is that they have The problem for many teachers is that they have no schema with which to process the data and no schema with which to process the data and make systematic judgements.make systematic judgements.


Where are we now?Where are we now?

Stiggins (2001) told Stiggins (2001) told ““a story of disregard by policy a story of disregard by policy makers, school leaders, and the measurement makers, school leaders, and the measurement communitycommunity”” for classroom assessment that has for classroom assessment that has ““kept classroom assessment from even kept classroom assessment from even approximating its potentialapproximating its potential”” (p.5)(p.5)

Have things changed since then?Have things changed since then?

In the measurement community and in language In the measurement community and in language testing/assessment, there are many signs of small testing/assessment, there are many signs of small changes.changes.


Classroom language assessment Classroom language assessment and politicsand politics

It is policy makers, school leaders, and to It is policy makers, school leaders, and to some extent parents who are slower to some extent parents who are slower to change.change.

There There areare positives: the Hong Kong SBA positives: the Hong Kong SBA now spreading to Singapore and Brunei; now spreading to Singapore and Brunei; European initiatives such as DIALANG and European initiatives such as DIALANG and the European Language Portfolio. the European Language Portfolio.

Portfolio assessment of writing in many Portfolio assessment of writing in many countries.countries.

ButBut……


Who makes Who makes change change

happen?happen?


ReRe--defining classroomdefining classroom--based based assessmentassessment

“…“… any reflection by teachers (and/or learners) on the qualities any reflection by teachers (and/or learners) on the qualities of a learnerof a learner’’s (or group of learnerss (or group of learners’’) work which has the potential ) work which has the potential for the use of that information by teachers (and/or learners) fofor the use of that information by teachers (and/or learners) for r teaching, learning (feedback), reporting, management or teaching, learning (feedback), reporting, management or socialization purposes.socialization purposes.””

“… the ‘assessment opportunity’ (‘any actions, interactions or artefacts (planned or unplanned, deliberate or unconscious, explicit or embedded) which have the potential to provide information on the qualities of a learner’s (or group of learners’) performance’), enables consideration of the more intuitive forms of teacher decision-making in CBA.”(Hill & McNamara 2012: 396)(Hill & McNamara 2012: 396)

Copyright Liz Copyright Liz HampHamp--Lyons 2013Lyons 2013 3333

Some ReferencesSome References

Black, P. & D. Black, P. & D. WiliamWiliam. (1998). Assessment and . (1998). Assessment and classroom learning. classroom learning. Assessment in Education, 5Assessment in Education, 5,1: 7,1: 7--7373..

DarlingDarling--Hammond. L. (2010). PerformanceHammond. L. (2010). Performance--based based assessment and educational equity.assessment and educational equity. Harvard Educational Harvard Educational Review,Review, 6464,1: 5,1: 5--31.31.

Hill, K. & T. McNamara. (2012). Developing a Hill, K. & T. McNamara. (2012). Developing a comprehensive, empirically based research framework comprehensive, empirically based research framework for classroomfor classroom--basedbased assessment. Language Testing.assessment. Language Testing.

ReaRea--DickinsDickins, P. (2007).Classroom, P. (2007).Classroom--based assessment: based assessment: Perils and pitfalls. In Cummins & Davison Eds. Perils and pitfalls. In Cummins & Davison Eds. International Handbook of ELT.International Handbook of ELT.

Yin, M. (2010). Understanding classroom language Yin, M. (2010). Understanding classroom language assessment through teacher thinking research. assessment through teacher thinking research. LAQ 7LAQ 7, , 2: 1762: 176--195.195.

Documents

The challenge of classroom classroom--based based assessment … Lyons The... · Ethical principles Classroom assessment must be fair. ... how well the assessment practices are interwoven