BERA

Developing Formative Assessment in the Classroom: Using Action Research to Explore andModify TheoryAuthor(s): Harry Torrance and John PryorSource: British Educational Research Journal, Vol. 27, No. 5 (Dec., 2001), pp. 615-631Published by: Taylor & Francis, Ltd. on behalf of BERAStable URL: http://www.jstor.org/stable/1501956Accessed: 03/09/2010 12:51

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/action/showPublisher?publisherCode=taylorfrancis.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

Taylor & Francis, Ltd. and BERA are collaborating with JSTOR to digitize, preserve and extend access toBritish Educational Research Journal.

http://www.jstor.org

British Educational Research Journal, Vol. 27, No. 5, 2001

Developing Formative Assessment in the

Classroom: using action research to explore and modify theory

HARRY TORRANCE & JOHN PRYOR, University of Sussex [1]

ABSTRACT This article reports the outcomes of a research project designed to investi- gate and develop formative classroom assessment in primary schools. The project was a collaborative one, involving two university-based researchers and a team of teacher- researchers. The aims were to build on basic research already carried out by the university researchers by investigating the issues from a more practical and applied perspective; consider how a collaborative action research approach to the professional development of teachers might be used to bring about changes in classroom assessment practices; and provide a basis for the further development and refinement of theory on formative assessment. The article reports on changes in classroom practice, particularly involving the clarification and communication of assessment criteria to pupils, and on the processes by which this came about.

Introduction

The continuing need to develop the potential of classroom assessment to support learning has recently been stressed by a number of researchers in the field (Assessment Reform Group, 1999). In particular, Black & Wiliam (1998, p. 61) have called for research which supports teachers 'in trying to establish new practices in formative assessment'. This article describes some of the outcomes of a research project designed to do just that-to investigate and develop formative classroom assessment in English primary schools. The project was a collaborative one, involving two university-based researchers and a team of teacher-researchers. There were two central aims: the first focused on classroom assessment issues and sought to build on basic research already carried out by the university researchers (Torrance & Pryor, 1998). The intention was to engage critically with the conclusions of this research by investigating the issues from a more practical and applied perspective; in short, to put the ideas generated by basic research to the test of practice. At the same time, the project sought to address a second aim, Received 28 September 2000; resubmitted 29 January 2001; accepted 16 March 2001.

ISSN 0141-1926 (print)/ISSN 1469-3518 (online)/01/050615-17 ?2001 British Educational Research Association DOI: 10.1080/01411920120095780

616 H. Torrance & J. Prvor

considering how a collaborative action research approach to the professional develop- ment of teachers might be used to bring about changes in classroom assessment practices. The focus of this article is on classroom assessment issues, but we also refer to the design and conduct of the action research, in order to explain overall project development and indicate the full range of outcomes.

The project, Investigating and Developing Formative Teacher Assessment in Primarn' Schools, was known more colloquially within the research team as 'Primary Response' [2]. The work grew out of a previous research project, Teacher Assessment at Key Stage 1 (the 'TASK Project' [3]). A brief initial summary of the previous 'basic' research is provided in order to describe the starting point of the project reported here. TASK was basic research grounded in classroom observation, which yielded a description and analysis of teachers' informal assessment practices in infant classrooms (i.e. ages 5-7 years). It explored the extent to which teachers and pupils shared an understanding of the nature and purpose of assessment 'events', drawing on and integrating a number of different theoretical perspectives. It engaged with the literature on formative and authentic assessment (e.g. Gifford & O'Connor, 1992; Gipps, 1994; Graue, 1993), in

particular on the ways in which routine classroom assessment might be integrated with pedagogy to maximise its formative potential in promoting learning. Neo-Vygotskian social constructivist approaches to learning (e.g. Newman et al., 1989) provided a basis for analysing assessment practices which might embody 'scaffolding' and thus hold the potential for making feedback more formative in a way that the matched teachers' espoused intentions. The fine-grained description and analysis of classroom interaction, on which the argument was based, was effected by paying close attention to linguistic structures (Barnes, 1976; Mehan, 1979; Edwards & Mercer, 1987). These perspectives were important in demonstrating that in suitable social contexts, forms of language arose, which departed from the usual patterns of classroom discourse. The research suggested that when this happened, there was more opportunity for enhanced student learning. The effect of feedback on pupils' motivation and goals also formed an important aspect of the work (Dweck, 1989; Ames & Ames, 1984/89; Urdan & Maehr, 1995) and insights from this body of work were used to critique some of the more optimistic claims made for the formative impact of assessment. Findings indicated that the effectiveness of formative classroom assessment in helping children to improve their work could not be assumed, and that there were great differences between children in the same class, dependent on their perceptions of the implicit social rules of the classroom and their orientation to achievement goals (cf. also Bloome & Willett, 1991; Kreisberg, 1992). Indeed, the research as a whole concluded that attention to the social construction and accomplishment of classroom assessment was a prerequisite for any systematic attempt to improve the quality of interaction and the positive impact of formative assessment on learning (Torrance & Pryor, 1998).

These different aspects of the basic research were brought together to create a model of classroom assessment as an intersubjective social process situated in, and accom- plished by, interaction between students and teachers. In particular, TASK identified two 'ideal-typical' approaches to formative assessment, which nevertheless were not necess- arily mutually exclusive in practice: one 'convergent', the other 'divergent' (see Fig. 1). These ideal types seemed to be associated with teachers' differing views of learning and of the relationship of assessment to the process of intervening to support learning, and might be said to represent a continuum of possibilities for classroom teachers. In conv'ergent assessment, the important thing is to find out if the learner knows, under- stands or can do a predetermined thing. It is characterised by detailed planning, and is

Developing Formative Assessment in the Classroom 617

CONVERGENT ASSESSMENT

Assessment which aims to discover if the learner knows, understands or can do a predetermined thing. This is characterised by:

Practical Implications a. precise planning and an intention to stick to it;

b. tick lists and can-do statements;

c. an analysis of the interaction of the learner and the curriculum from the point of view of the curriculum;

d. closed or pseudo-open questioning and tasks; e. a focus on contrasting errors with correct

responses;

f. judgmental or quantitative evaluation;

g. involvement of the student as recipient of assessments.

Theoretical Implications h . a behaviourist view of learning; i. an intention to teach or assess the next

predetermined thing in a linear progression; j. a view of assessment as accomplished by the

teacher.

This view of assessment might be seen less as formative assessment, rather as repeated summative assessment or continuous assessment.

DIVERGENT ASSESSMENT

Assessment which aims to discover what the learner knows, understands or can do. This is characterised by:

Practical Implications a. flexible planning or complex planning which

incorporates alternatives; b. open forms of recording (narrative, quotations

etc.); c. an analysis of the interaction of the learner and

the curriculum from the point of view both of the learner and of the curriculum;

d. open questioning and tasks; e. a focus on miscues - aspects of learner's work

which yield insights into their current understanding, and on prompting metacognition.

f. descriptive rather than purely judgmental evaluation;

g. involvement of the student as initiator of assessments as well as recipient.

Theoretical Implications h a social constructivist view of learning i. an intention to teach in the zone of proximal

development; j. a view of assessment as accomplished jointly by

the teacher and the student.

This view of assessment could be said to attend more closely to contemporary theories of learning and accept the complexity of formative assessment.

FIG 1. Convergent and divergent classroom assessment.

generally accomplished by closed or pseudo-open questioning and tasks. Here the interaction of the learner with the curriculum is seen from the point of view of the curriculum. The theoretical origins of such an approach would appear at least implicitly to be behaviourist, deriving from mastery-learning models and involving assessment of the learner by the teacher. Divergent assessment, on the other hand, emphasises the learner's understanding rather than the agenda of the assessor. Here, the important thing is to discover what the learner knows, understands and can do. It is characterised by less detailed planning, where open questioning and tasks are of more relevance. The implications of divergent teacher assessment are that a constructivist view of learning is

adopted, with an intention to teach in the zone of proximal development (Vygotsky, 1986). As a result, assessment is seen as accomplished jointly by the teacher and the student, and oriented more to future development rather than measurement of past or current achievement.

The 'Primary Response' Project Design

A key issue that emerged towards the end of the TASK project was how to integrate the insights derived from basic research with the development of practice. The Primary Response research and development project was designed to carry forward and test these insights in a more applied programme of work. We were interested in investigating through collaborative action research with teachers how emerging theoretical understand- ings of formative assessment can be translated into practical classroom strategies in primary schools. The overall intention of the research was to test the validity and utility

618 H. Torrance & J. Pryor

of the concepts of convergent and divergent assessment with a group of collaborating 'teacher-researchers', while at the same encouraging them to use the concepts to investigate and reflect upon their own classroom assessment practices in order to improve them. Thus, we were concerned with whether, and to what extent, teachers could develop their classroom assessment practices and integrate them into a more self-consciously articulated model of classroom pedagogy, in order to test the claims of formative assessment in ordinary classroom settings. We also wanted to explore whether, and to what extent, an 'action research' approach could help to accomplish the process of development.

A central problem of any application of research to teaching is the potential conflict between theoretical and practical wisdom. Brown & McIntyre (1993) note that innova- tions are rarely accepted by teachers because they are viewed as impractical; that is, they do not accord with the teachers' own tried and tested ways of teaching. Drawing on the work of Elliott (1991), Fenstermacher (1986) and others, we sought to engage with teachers' premises about learning and assessment, through a programme of discussion and an initial classroom reconnaissance. A further process of action research was then set up to enable them to develop 'practical arguments' (Fenstermacher, 1986). Whilst trying to avoid the oversimplification of viewing theory as non-practical and practice as non-theoretical (cf. O'Hanlon, 1994), the project nevertheless sought to recognise these discursive differences and to maintain a dialogue between them.

Thus, the research team comprised teacher-researchers (TRs), conducting research on their own classroom practices, and university researchers who combined additional interviewing and classroom observation with the orchestration of overall team develop- ment and direction. The 'action research' approach involved discussing with the TRs their implicit theories of learning and assessment whilst also supporting them in investigating practice within their own classrooms and thus developing knowledge about particular events and practices-the engagement with premises. Only after they had reflected and reported on this stage were they engaged more closely as 'critical consumers' (Hollingsworth, 1997, p. 486) in the process of interrogating public knowl- edge about formative classroom assessment processes and in generating such knowledge themselves. Thus, the project sought to explore the extent to which such an approach to knowledge creation and professional development could be utilised in this field (cf. Elliott, 1991, 1998; Altrichter et al., 1993).

In the spring of 1997, we recruited a team of TRs in collaboration with a neighbouring local education authority (LEA). A 'flyer' was sent to all primary schools in the county, resulting in an initial meeting with 11 prospective TRs in May 1997 (two of whom had been involved with the previous TASK project and wished to continue to be involved). The only selection criteria were the TRs' interest in the project and their headteacher's support: thus, the team comprised a group of very 'ordinary' primary school teachers (10 female, I male).

The action research went through two cycles or phases: May-December 1997 (phase 1), and January-August 1998 (phase 2). Data were gathered by the TRs through the audio and video recording of assessment interactions in their classrooms and by keeping research diaries and examples of students' work. A 'core' of seven TRs completed seven initial investigations of their classroom assessment practices (phase 1) and five of these went on to explore specific interventions and new approaches (phase 2). In total, 12 reports were completed, totalling c. 120,000 words (12 x c. 10,000 words) [4]. Thus, of the 11 TRs originally recruited, five completed two cycles of research activity resulting in project reports (completed in January 1998 and August 1998); two completed one

Developing Formative Assessment in the Classroom 619

cycle and report (one in January 1998, the other in August 1998); and four withdrew

during the first 8 months of the project without producing any written reports-either through pressure of work at school (e.g. promotion) or through ill health.

In addition to carrying out the action research projects in schools, and being supported in this by visits from the university-based researchers, the team met 17 times during the period May 1997-December 1998 in half-day, whole-day, and occasionally in 'twilight' (after school) sessions, and for a 2-day residential evaluation and writing workshop. Supply cover [5] was available in the budget to facilitate this programme of meetings. The structure and sequencing of the meetings moved from:

"* introductory input by the university researchers, raising issues of assessment and learning;

"* through discussions of the logic and methods of action research; "* to the design of specific investigations; "* to the reporting of progress and discussion of data; "* to the presentation of emerging findings-both substantive and methodological-and

more explicit interrogation of the problems and possibilities of formative classroom assessment (see Fig. 4).

As a result of this activity, we were able to build up a data set comprising:

"* TRs' reports, including classroom data and TRs' analysis of them; "* TR responses to various structured tasks during the 2-day evaluation and writing

workshop; "* field notes from and audio tape-recordings of team meetings; "* interviews with TRs (including those who did not complete reports); "* field notes from and audio and video tape-recordings of TR classrooms made by the

university researchers.

All TR reports were circulated for all members of the team to read. The need for internal confidentiality in the conduct and discussion of the action research studies, combined with a commitment to move to public reporting, was ensconced in an ethical agreement.

One further point must be highlighted which in a sense blurs the boundaries between 'methods' and 'results'. A central part of the methodological approach was that the development of the TRs' practical arguments should be enabled and facilitated through the introduction of key theoretical resources. Phase 1 of the project contributed to the development of some of these resources, which were then able to be tested and further developed in phase 2. During phase 1, the university researchers were also working on the manuscript of a book of the previous project (Torrance & Pryor, 1998). Partly as a result of phase 1 discussions, we used the data from the basic research project to develop a descriptive and analytic framework of the processes of formative assessment, which formed a significant part of the concluding chapter of the book (see Fig. 2). At the same time as the TRs' first reports were circulated to all team members (January 1998), the final manuscript of the book was also made available to TRs. The analytic framework (Fig. 2) was used by team members to describe and analyse their own practices. Although subsequently viewed more critically, especially with respect to how it was necessary to reintegrate aspects of the framework into a more holistic model of classroom assessment, the utility of this framework has become one of the key findings of the overall project. Thus, the team used, tested and modified both the original concepts of convergent and divergent assessment and the analytic framework which appeared in Torrance & Pryor (1998). Subsequent modification of the framework is

620 H. Torrance & J. Prvor

Description Possible teacher intentions Possible positive effect for student

A T communicates task criteria (what Communicating goals and success Understanding of task and principles has to be done in order to complete criteria; ensuring work is on target; behind it. the task) or negotiates them with S. adjusting pace of work.

B T communicates quality criteria Enhancement of quality of future Understanding of notions of quality (what has to be done to do the task work; promotion of greater S to aid future self monitoring. well) or negotiates them with S. independence.

C T observes S at work (process). Gain in understanding of why/how Enhanced motivation due to T's student has approached or achieved attention. task.

D T examines work done (product). Gain in understanding of what S has Enhanced motivation due to T's done. attention.

E T asks principled question (seeks to Insight into S's knowledge, Rehearsal of knowledge, elicit evidence of what S knows, understanding or skills, understanding, or skills; articulation understands or can do: substance). S of understanding to realise responds. understanding.

F T asks for clarification about what Gain in understanding of what S has Re-articulation of understanding; has been done, is being done or will done and of S's understanding of the enhanced self-awareness and skills of be done: process; S replies, task. summary, reflection, prediction,

speculation.

G T questions S about how and why Gain in understanding of why/how the Articulation of thinking-about- specific action has been taken student has approached or achieved thinking; deepened understanding and (meta-process and metacognitive task. 'handover'. questioning). S responds. Promotion of deepened understanding

and 'handover'.

H T critiques a particular aspect of the Enhancement of quality of future Articulating and interrogating quality work or invites S to do so. work; promotion of greater criteria; enhanced understanding of

independence. quality issues; practice in self- monitoring.

I T supplies information, corrects, or Communication of alternative or more Enhancement of knowledge and/or makes a counter-suggestion. acceptable product. understanding.

1 T gives and/or discusses evaluative Influence on S's attributions and Enhanced motivation and self-worth feedback on work done with respect therefore motivation of S for further when realised in a context of to: task, and/or effort and/or work. empowerment; development of aptitude/capability (possibly with learning goals. reference to past or future achievement: ipsative).

K T suggests or negotiates with S what Insight into ways forward for Insight in ways to continue working to do next. immediate further teaching of and learning. Deepening of

individual; refocusing S on curricular understanding of process/principle. goals.

L T suggests or negotiates with S what Insight into ways forward for Deepening of understanding of to do next time. planning of group activities, principle/process.

M T assigns mark, grade or summary Information for summative Information about present judgement on the quality of this assessment; communication of quality achievement with respect to longer piece of work or negotiates an criteria; teaching/modelling skills of term goals. agreed one with S. assessment for self-assessment.

N T rewards or punishes student, or Improvement or maintenance of Enhanced motivation. demonstrates approval/ disapproval. relationship with student;

enhancement of motivation.

FIG 2. A descriptive and analytic framework of the processes of formative classroom assessment (adapted from Torrance & Pryor, 1998).

reported and discussed below, as this modification represents a key aspect of the transformative process from classroom research to more self-conscious and theoretically- informed classroom practice.

Substantive Findings with Respect to Developing Formative Classroom Assessment

Members of the team began with varying degrees of experience and expertise and were

working in different school environments, while also carrying different responsibilities. General findings across the team must therefore be treated with due caution. Neverthe- less, common issues can be identified-quite strikingly in some respects. Initial team

Developing Formative Assessment in the Classroom 621

meetings revealed that the TRs were positive about developing formative assessment (they would hardly have volunteered for the work otherwise, and most were assessment coordinators in their schools) but had a fairly narrow view of what constituted 'assessment' and the legitimate teacher role in it. While assessment was no longer conceptualised purely in terms of officially sanctioned tests, the TRs still viewed it as an essentially formal requirement. Assessment, even within the regular routines of the classroom, was seen as an additional task, separate from teaching. Formative assessment, in so far as it carried any meaning, was perceived incrementally in terms of a product to be obtained from students and then used for the planning of teaching. However, how the TRs might consistently do this, i.e. actually use assessment data to help plan for the needs of all the children in their class, remained unclear to them.

Thus, there was little sense at this stage that classroom assessment might be a process to be developed as part of their teaching repertoire. When the notions of convergent and divergent assessment were introduced, convergent assessment was immediately recog- nised as representing mainstream classroom practice, though not necessarily practice that the TRs were happy with, or had willingly developed-rather, it was practice that they felt obliged to follow by government policy. Also, perhaps because they felt convergent practice was largely imposed, TRs initially had great difficulty in relating assessment to theories of learning, or indeed in articulating personal theories of learning that they worked with in the classroom. They were more comfortable in dealing with concrete examples drawn from video data. Ideas were expressed through descriptions of class- room organisation and school policy, e.g. taken-for-granted notions of 'ability' as expressed through 'ability groupings' at different classroom tables were identified and explored; the practice of developing 'independence' in children through encouraging them to complete set tasks before then going on to choose others led to discussion of why independent engagement with tasks was considered to be important; and school reward systems as expressed through merit marks and 'smiley' faces led to discussion of the impact of rewards on learning [6].

With respect to the action research process, phase 1 work was devoted to reconnais- sance-to the TRs investigating their own classroom teaching and assessment practices by making audio and video tape-recording of interaction. Thus, the TRs produced data-rich accounts of how they framed and managed classroom activities through their instructional interventions. A significant discovery for them was that in many cases their teaching seemed to close down opportunities for exploring student understanding rather than opening them up. A second consistent finding in this opening phase, as the TRs monitored their ordinary classroom practices, was that very little of the purpose of classroom activities was made clear to students, far less what would count as success in terms of quality (as opposed to straightforward task completion). The TRs were surprised and in some respects a little embarrassed to discover how little they focused on learning goals as opposed to behavioural goals and classroom management. Thus, a key finding of the overall project is that teachers need the opportunity to monitor and reflect on their own classroom practices-to investigate them in detail-before being ready to then think about how best to develop more principled intervention strategies. In Fenstermacher's (1986) terms, teachers need to examine their premises before attempting the process of argumentation. However, by the end of this opening period, the TRs were coming to regard formative classroom assessment as a constituent part of their pedagogy:

I'm taking much more notice ... looking at how they're going about their work, how they are understanding it while they're doing it ... I don't think I

622 H. Torrance & J. Pryor

was very good before at asking open-ended questions of children ... I was sort of testing what they could do all the time. (TR6) [7]

From this opening phase, a number of concerns were identified for investigation in the second cycle of action research:

(i) the need to make more explicit what was the purpose of certain activities and what would count as doing them well (i.e. task and quality criteria);

(ii) the need to respond more flexibly to students in the classroom and think about developing a more 'divergent' approach to formative assessment; in particular, this was manifested in a concern to develop different approaches to questioning, observation of the students at work, and feedback.

The first phase had heightened the TRs' awareness of the complexity of classroom interactions, and a great deal of what they routinely did was now called into question. The framework introduced at the beginning of phase 2 (Fig. 2) provided an analytic account of the range of possible activities included in formative classroom assessment and a vocabulary with which to interrogate, express and organise their data and subsequent ideas for development. Conceptual distinctions, such as that between 'task criteria' and 'quality criteria' came to be regarded as important analytic tools. They not only provided a means to describe and explain the areas of their practice on which they had already focused, but also brought into the open other elements of their intuitive and implicit practice. Thus, the framework was used in parallel with data which they themselves had gathered in their own classrooms, as a device for disinterring their practice and subjecting it to detailed analysis in identifying the frequency and importance of each of the different processes.

In the second phase of action research, TRs began to operationalise changes in their practice. Analysis thus gave way to synthesis. Having identified aspects of classroom assessment that they wished to improve, they began to focus on those categories within the framework that might help them to do so. Greater importance was now attached to the way that the processes might be integrated and manipulated and to considering how convergent and divergent approaches to each might be exploited according to the constraints of the particular context.

However, the framework also came to be viewed more critically. Although TRs were still aware of the conceptual distinctions between the different processes, and could relate them to examples of their own practice, they began to point out that, in practical situations, these processes were often embedded one within another, or occurred in linked sequences or progressions. Key elements of the analytic categories became reintegrated into new, more self-consciously theorised practice. The categories of the initial framework were a useful device for conceptualisation and mental referencing, but the actual accomplishment of formative assessment involved movement between more synthetic and permeable categories. Thus, more open-ended questioning and observation of the students at work were combined with carefully judged feedback. In particular, the clarifying through dialogue of criteria with respect to both task and quality became the focal point and purpose of the interaction. While the need to establish learning goals and criteria for judgement, and communicate them clearly to students, had come through very strongly in the first phase of the action research, actually accomplishing this came to be perceived as a continuing dynamic and interactive process. Students' understanding of goals and criteria could not be taken for granted simply because they were now being made explicit, i.e. being stated by the TRs in a way in which they might not have been

Developing Formative Assessment in the Classroom 623

QUESTIONING: 'HELPING' AS WELL AS 'TESTING' QUESTIONS

OBSERVATION OF PROCESS AND PRODUCTS

FMAKING' TASK AND

QUALITY CRITERIA

•EXPLICITA

FEEDBACK AND

JUDGEMENT

FIG 3. Formative assessment in practice.

stated before (e.g. at the start of a task, or lesson, or sequence of lessons). Goals and criteria also had to be pursued and continually refined and re-established through interaction (in effect, through scaffolding learning). Fig. 3 has been constructed to describe this process.

Fig. 3 attempts to represent this interactive process, reintegrating into a practical classroom model the analytic categories included in the original framework (Fig. 2), and it does so using language generated by the TRs ('helping' as well as 'testing' questions). In particular, it places clarity of criteria at the core of classroom practice, but established dynamically, through the interaction of questioning, observation and feedback, rather than simply being stated or asserted, as through a more transmission-oriented model.

The presentation and discussion of task and quality criteria was seen by TRs as being at the heart of the process, crucial to the development of more transparent social relations and pedagogic activities in the classroom. This was especially significant in the light of the problematic nature of much classroom interaction (see also under questioning, below). The distinction between task criteria and quality criteria might sometimes blur in practice, but what was important was that both were referred to and that criterion- setting always included reference to quality criteria. In addition, task criteria had to relate to both the immediate task at hand and the more general 'social rules' of the classroom-teacher expectations of behaviour and implicit rule-following which are often assumed but rarely taught (e.g. re independence or cooperation in small groups). Making such practices visible (discussing how pairs or small groups could and should work together, for example), over and above the procedures for the accomplishment of any particular substantive task, came to be seen as extremely important in establishing the context for classroom assessment: knowing 'not just what to do, but also the purpose of the task' (TRI) and reaching 'some shared understanding on goals and aims' (TR5). However, referring to criteria consistently was not seen as easy, and, especially to begin

624 H. Torrance & J. Pryor

with, was seen to be very time-consuming. The TRs also noted, however, that time spent articulating criteria at the beginning of an activity could mean less 'trouble-shooting' later as they became a reference point for feedback which both built on and underlined their importance. In addition, it was important that issues of quality were well focused: TR4 mentioned the problem of overemphasising simply completing a task rather than doing it well, and TR1 saw as problematic the possibility that teachers might be specifying aspects of quality 'beyond the attainment of certain children'. Thus, it was suggested that the expression of quality criteria in particular had to be accomplished more by interaction with individual students through questioning, rather than by articulation to the whole class. Indeed, to reiterate, an important insight into the setting of criteria is that the process is not just confined to the start of a lesson, but rather is achieved through dialogue during a process of literal or metaphorical 'drafting'. Providing opportunities to improve an initial attempt at a task both extends the learning event and creates the conditions for continuous clarification of criteria. This is especially the case when divergent approaches are used.

In turn, the TRs noted that developing different forms of questioning, and using them in different ways (categories E, F and G in Fig. 2) was most important, but that involvement in the project had rendered this problematic. For TR1, this was primarily an issue of power, which potentially made questions seem threatening. Her intervention was to adopt a more divergent approach, where questions were designed to 'point the way', contributing over time to developing 'the confidence in children to see questioning as helpful and non-threatening'. TR2 also noted that children in her class made a distinction between 'helping' questions as opposed to 'testing' questions (Fig. 3) and articulated a need for greater transparency of classroom processes whereby the differing intentions behind questions are made more explicit. TR6 and TR4 set up specific tasks that involved children in questioning each other. TR4 and TR3 suggested that encourag- ing children to talk more about their interpretation of questions had presented a useful way forward for them. However, there was a contrast between these two teachers in the way they saw their practice changing: whilst TR3 found herself 'more confident in doing this in an ad hoc manner rather than having to pre-plan' her questioning, TR4 talked in terms of the amount of 'thought that goes into my presentation' and being more careful of the way that she might influence children. Thus, the same issue was explicitly recognised by these two teachers, but incorporated into their practice in different ways. Similarly, TRI talked of needing 'to develop the ability to stand back and let the children talk and clarify without interrupting'. Questioning was also seen as problematic in that a less confident child might interpret a clarifying question as an indication that they had not got the original answer right. Clarifying questions might be perceived as intimidatory (TR7) and their effectiveness thus depends on 'skilful and sensitive questioning' (TR5) and on creating situations where children see the purpose of such questions as being a stimulus to thought rather than a means of disciplining them.

Observation was the foundation of the TRs' formative assessment and a fundamental way in which they obtained information about what children know, understand and can do. However, it can also form part of a rather ritualistic response to problems of classroom assessment; thus, the focus of the observation--what activity is being observed and for what reason-is more important than the managerial act of monitoring classroom behaviour and apparent time on task. Hence, we report the importance of observation here, after, rather than before, questioning. Indeed, a key practice with respect to focus and understanding is the combination of observation with questioning ('I gain most of my formative data observing and talking to children about their work'

Developing Formative Assessment in the Classroom 625

[TR4]). Thus, while observation was often held to be the 'initial stage' of the process (TR5), preceding various other forms of interaction, it was linked very closely to questioning, and one effect of thinking more about observation was to change the quality of the questioning. The important thing for the TRs was to avoid rushing to make a judgement. TR1 described the process as taking 'time to actually observe, not just see, what's going on and then intervene'. But it was also agreed that this was not just a question of 'standing back' and simply 'bracketing' the contingencies of the classroom, but of seeing a real purpose and intellectual challenge in observing children at work, which came from a more divergent approach to assessment. As TR6 commented:

it's the awareness, I have plans, but now I'm more fluid in my lessons and not just judging it there and then ... I'm deliberately finding out what they know by talking to the children.

Moreover, 'watching how children approach a task is revealing. It's more motivating for me, as well as children' (TR3).

Feedback, then, had to be carefully judged in terms of impact on the process of completing the immediate task at hand, and carrying forward understandings of sub- stance and process to future activities. The project highlighted the ambivalent attitude of the TRs towards making and communicating overall judgements about work. Judge- ments were seen to place emphasis on products rather than process, but TR4 pointed out that, perhaps paradoxically, the teacher's judgement is very important in indicating the level of achievement needed to 'build the reference framework for self-assessment'. Here, we return to the issue of clarifying and communicating quality criteria and focusing the students' attention on the intended learning goals of the activity, rather than on performance goals (cf. Dweck, 1989). Older children still seemed to want a summary grade, especially as they approached the National Curriculum Key Stage 2 tests, a point noted by both TR3 and TR2, but this effect seemed to be diminished when they had themselves been asked to reflect on their work. TRI said she tried not to make overall judgements, since being in the project, except in combination with more focused critique, but noted that it is impossible to give a critique without at least an implicit judgement. An interesting insight into the balance between focused and general judgements was that when discussing the issue as a group, the teachers reported that they gave more overall judgements (of the 'good', 'satisfactory' variety) in subject areas where they were less confident and less able to focus on particular difficulties ('When you are not confident of the subject matter you tend to give less focused judgements' [8]). Additionally, TRs stressed that when rewarding, it was important to take into consideration the differences between children, though this may not fit easily with whole-school policies. Certainly, the TRs recognised that any differentiation had to be seen to be fair. Older children, in particular, were aware of each other's attainment. This made it important to emphasise ipsative comparisons and to be less reticent about saying when a child had not tried hard enough. Both approval and disapproval could be useful in 'supporting the ethos of the classroom' and maintaining a 'positive structure of expectations' (TR4), though in order for disapproval of an individual to work effectively it had to happen within a context where there was a generally positive atmosphere.

Methodological Findings with Respect to Action Research and Project Design

The analytic framework and the input on assessment and learning, particularly as manifested in discussions about convergent and divergent assessment, were appreciated

626 H. Torrance & J. Pryor

and utilised by the TRs. Also, there was a dynamic modification of the position and understandings of both the TRs and the university researchers as the project progressed. Overall, however, it was the opportunity to gather data on themselves, supported by visiting researchers and by the use of technology such as audio and video tape-recording, which was of fundamental importance to the TRs in coming to understand the complexities of teacher-student interaction and accomplishing change in their own classrooms:

I found it really interesting. It taught me a lot about the children in the class and my teaching ... Until you've analysed your own data, you can't really take it in, even though you're reading about someone else doing something. (TRI)

it's not until you've done the action research and sat and found out what's going on that ... it is clearer. (TR3)

Our data suggest that in the reconnaissance phase, stimulated by the project's focus on teaching and learning and the introduction of the heuristic concepts of convergent and divergent assessment, the TRs had moved beyond routine self-monitoring to what Elliott (1998, p. 186) describes as 'the conscious self-monitoring of practice'. However, in order to progress, towards what he describes (after Giddens, 1984) as the development of 'discursive consciousness', they required additional intellectual resources. These were supplied by the analytical framework (Fig. 2). Not only did it provide a vocabulary for analysis, but did so in the context of their own perceived need for making sense of what they had experienced. Lengthy and repeated discussion, together with exploratory data collection, had encouraged them to examine their own tacit theories of learning and assessment and resulted in a changing of their premises about formative assessment. The framework enabled them initially to reappraise their practices and then gave them a starting point for their research interventions. This second phase consisted of the development and implementation of new classroom practices where analysis was less important than the need to integrate and synthesise the various categories of the framework (in Fenstermacher's terms, the construction of a practical argument). The result was the changed, more self-consciously theorised way of approaching formative assessment, which was informed by the framework but which in practice conformed more to the process shown in Fig. 3. Our suspicion would be, however, that simply starting with the more integrated model would not be so productive, since it would hide as much as it revealed of the complexity of the process. Conducting the action research and generating the model was an important transformational act in moving from theory to practice.

The TRs were very positive about the support that the team provided for them and about the project's overall approach. In an evaluation exercise during the residential workshop, 'discussion at Primary Response meetings', 'own reflection on data col- lected', and 'looking at data at PR meetings' were ranked first, second and fourth in importance as activities with respect to influencing professional understanding and practice [9]. Furthermore TRs considered that their ability to engage with students in formative assessment was much enhanced. Thus, we can claim that our model for collaborative action research (Fig. 4), where theoretical knowledge is not so much transmitted to teachers as mediated and transformed through practical arguments, was successful as an exercise in professional development.

The TRs claimed that, by doing research, they become a different kind of teacher (cf. Wilson, 1995) and, depending on the degree to which this change was acceptable to

Engagement with premises

Changed premises Practical argumentation

Changed praxis

Discussion about learning & assessment incl. convergent/ divergent

Analysis of classroom data

Implementation of framework

Transformation of framework

Classroom reconnaissance

Presentation of conceptual framework

Evaluation and monitoring of practice

FIG 4. Model for collaborative action by university and teacher researchers.

others within the school, it was more or less comfortable and sustainable (cf. Hausfather, 1997).

However, claims are also made about action research as a means of giving 'voice' to practitioners (Winter, 1998). A key question is, therefore, to what extent and in what

ways this model might also be suitable for generating and reporting public knowledge about the substantive topic of concern (cf. Elliott et al., 1996). To date, this has remained the responsibility of the university researchers. Despite the production of 12 TR reports, which can be accessed as public documents with the permission of the authors, and our intention to produce a co-authored book from this work, we have not been able to create a fully integrated and equal research team investigating the development of formative assessment. Rather, the team provided a support structure (albeit a very important one) for the pursuit of individual projects within an overall framework of investigation sustained by the university researchers. A variety of disparate reasons would seem to account for this, which we explore elsewhere (Torrance & Pryor, 1999). For present purposes, we can simply note that although TRs were integrated into a supportive team, they remained immersed in their own projects. Their task was therefore 'generating and testing new forms of action for realising their aspirations' (Elliott, 1994; p. 136). We can claim that the project allowed TRs the opportunity to 'problematize their educational practices by reflecting on their underlying meanings' (Elliott et al., 1996). However, the generation of public knowledge, i.e. the distillation of findings, establishment of an overview and theorisation at a meta-level, remained the province of the university researchers. Rather than being seen as a weakness, however, this might be held to be a strength, since TRs lacked the resources (in particular those of time) to engage at this level. Successful collaboration was an outcome of the fact that neither TRs nor university researchers would have been able to accomplish what they did without each other: they had different agendas and priorities both at the level of role and of output.

Conclusions

Formative Assessment

The project confirmed that formative classroom assessment is a key theoretical and practical interface for teachers to engage in research and development on teaching and learning. This may be in part because formative assessment (as the set of practices developed by this project), especially in its divergent forms, is itself the aspect of pedagogy which most approximates to a research activity.

628 H. Torrance & J. Pryor

An initial starting point for the development of effective formative assessment is the

development of pedagogical self-awareness. This involves recognising and being cau- tious about cueing 'right answers' through routine, taken-for-granted teacher-student interaction.

Convergent and divergent assessment proved to be illuminating and provocative abstractions which helped the TRs to conceptualise their approaches to formative classroom assessment. TRs investigated divergent approaches more closely, because they came to recognise divergent assessment as potentially more powerful in fostering the social and intellectual conditions in the classroom which would lead to enhanced

learning. Nevertheless, they still argued that convergent approaches were important, and indeed, inevitable, given the convergence of the curriculum and constraints on teacher time. Indeed, coming to understand that they could develop and use a 'repertoire' of assessment strategies and practices was important in allowing the TRs to feel more secure in their explorations-getting away from the idea that somehow they had to

implement something completely new (divergent assessment). Rather, they used the

concept of divergent assessment as a heuristic device with which to explore and expand the boundaries of their classroom practice.

TRs found that the conceptual distinctions contained in the framework (Fig. 2) provided an extremely helpful vocabulary for describing their practice and subjecting it to detailed analysis; and that this process of analysis was important. However, sub-

sequent practical development required reintegration into a more holistic and dynamic model focusing on the progressive clarifying of criteria achieved through initial dis- cussion, questioning, observation, and feedback (Fig. 3).

Teachers need to clarify learning goals and task criteria not only by making clear the

purpose of and criteria for judgement of any individual task, but also by making clear the relationship of particular tasks to the overall management of the classroom-i.e.

making the 'social rules' of the classroom as transparent as possible, in addition to the

purpose and criteria of individual tasks. However, more crucially, quality criteria also have to be communicated to students.

These need to be discussed at the beginning of a task, but a continuing dialogue with

groups and individuals is also necessary as tasks are pursued and various forms of

drafting, both actual and metaphorical, take place. Students should be encouraged to

engage in this dialogue with each other as well as with the teacher. A variety of questions, intended to be perceived by students as 'helping' questions,

should be used to elicit understanding and guide progress. This is crucial for communi-

cating quality criteria and realising 'scaffolding' in action. Particularly useful forms of such questioning are elicitations which invite students to clarify and to reflect on their own thinking.

Observation should guide questioning, with teachers' interventions then being more informed and better focused. A rush to judgement should be avoided and interventions should be in the form of feedback related to the criteria. This feedback should identify clearly to what extent the task has been completed and to what extent and in what ways its quality could be improved. It will be enhanced when opportunities are provided for students to enter into dialogue and make their own judgements. Thus, clarity is achieved not so much through prescription as through continuing dialogue.

The Development of Professional Knowledge about Teaching, Learning and Assessment

For teachers to be able to develop new approaches to formative assessment and relate

Developing Formative Assessment in the Classroom 629

them to different theories of learning, they must be able to investigate and reflect upon their own classroom practices-particularly the way they question and give feedback to students. An action research approach seems particularly suited to high-quality develop- ment work on the interface between teaching, learning and assessment.

Gradual engagement with teachers' premises followed by the provision of an analytic framework proved a useful way of managing collaboration between university re- searchers and TRs and in brokering educational theory to classroom practitioners (Fig. 4). However, it is not enough simply to make intellectual resources available to teachers; they also have to want to use them. This was achieved through critical study of practice followed by thinking through the relevance of more flexible and integrated approaches to teaching and assessment. Once this was established, they were in a good position to further develop ideas through 'practical argumentation'.

At the substantive level Primary Response has contributed to both the practical development of formative assessment and its ongoing theorisation through its focus on classroom processes. However, some of the most interesting data generated by the project were student perceptions of the assessment process, and this is an aspect which could well be further developed. Research which looked in more detail at this area, investigating especially students' perceptions of the role and value of peer and self- assessment, and of 'helping' friends in the classroom might provide very valuable further understanding of formative assessment. Similarly, with respect to the future development of action research and teacher research, a key topic would be to investigate the conditions under which schools themselves might develop a 'research and development' capacity. Although, as we have already pointed out, the subject matter of the project was perhaps particularly amenable to the methodology adopted, there are grounds for believing that the model of collaboration which we used might be adapted for in-school development work. A similar combination of basic and applied research with respect to other aspects of the relationship between teaching, learning and assessment could prove fruitful. However, an important requirement of such work would be that teachers were in a position to take enough time for the initial exploration and description of practice, before going on to reflect on how such data could be used to underpin more self- consciously theorised practice.

Correspondence: Harry Torrance, Graduate Research Centre in Education, University of Sussex, Brighton BNl 9RG, UK; E-mail: h.torrance@sussex.ac.uk and/or j.b.pryor@ sussex.ac.uk

NOTES

[1] This article has been produced as a result of collaborative work with a number of teacher- researchers. Seven have produced data and written reports on which we draw here: Julie Collins, Jane Cowley, Janice Gill, Margaret Maiden, Sandie Piper, Susie Round and Sally Turner. Their contributions are not acknowledged separately, in this brief overview, but it is our intention to produce a book from the research which reflects and recognises these contributions more individu- ally. We are, of course, extremely indebted to all the hard work and intellectual contribution made to the project by the teacher-researchers.

[2] Supported by the UK Economic and Social Research Council (R000236860). [3] 'Teacher Assessment at Key Stage 1: accomplishing assessment in the classroom', funded by the

Economic and Social research Council (ref. number R000234668). See Torrance & Pryor (1998) for a full account.

[4] Master's-level (MA) credit was offered for successful submission of these reports, through the

630 H. Torrance & J. Pryor

University of Sussex Institute of Education Accreditation of Continuing Professional Development (ACPD) programme. Four of the TRs have subsequently gone on to produce MA dissertations for award of a full MA degree.

[5] i.e. payment was available to the TRs' schools to employ substitute teachers when the TRs were at

meetings. [6] There is also a methodological issue here, in that we were engaging with such issues quite early in

the project. Some of the TRs later stated that they felt intimidated by each other and the university environment (the experience of the teachers ranged from 3 years since qualification to over 20 years) and with hindsight, we might have paid more attention to developing the social and intellectual

integration of the team before attempting to explore intuitive understandings and use of learning theory. Thus, it is not that TRs didn't 'know' any theory; rather, they took time to feel comfortable

discussing it. [7] In this article, we have retained the practice of using numbers to denote individual TR contributions,

adopted originally for indexing individual contributions to the data set while maintaining anonymity in public reporting. As noted earlier, in the longer term, it is our intention to publish collaboratively and acknowledge individual contributions.

[8] Unattributed TR quotation from audiotape recording of team discussion. [9] Inputs from the university researchers ranked third.

REFERENCES

ALTRICHTER, H., POSCH, P. & SOMEKH, B. (1993) Teachers Investigate Their Work: an introduction to the methods of action research (London, Routledge).

AMES, C. & AMES, R. (1989) Research on Motivation in Education, vol. 3 (San Diego, CA, Academic Press).

AMES, R. & AMES, C. (1984) Research on Motivation in Education, vol. 1 (Orlando, FL, Academic Press).

ASSESSMENT REFORM GROUP (1999) Assessment for Learning: beyond the black box (Cambridge, University of Cambridge School of Education).

BARNES, D. (1976) From Curriculum to Communication (Harmondsworth, Penguin). BLACK, P. & WILLIAM, D. (1998) Assessment and classroom learning, Assessment in Education, 5, pp.

7-74. BLOOME, D. & WILLETT, J. (199 1) Toward a micropolitics of classroom interaction, in: J. BLASE (Ed.) The

Politics of Life in Schools, pp. 207-236 (Newbury Park, CA, Sage). BROWN, S. & MCINTYRE, D. (1993) Making Sense of Teaching (Philadelphia, PA, Open University Press). DWECK, C. (1989) Motivation, in: A. LESGOLD & R. GLASER (Eds) Foundations for a Psychology of

Education (Hillsdale, NJ, Erlbaum). EDWARDS, D. & MERCER, N. (1987) Common Knowledge: the development of understanding in the

classroom (London, Methuen). ELLIOTT, J. (1991) Action Research for Educational Change (Philadelphia, PA, Open University Press). ELLIOTT, J. (1994) Research on teachers' knowledge and action research, Educational Action Research,

2, pp. 133-137. ELLIOTT, J. (1998) The Curriculum Experiment: meeting the challenge of social change (Buckingham,

Open University Press). ELLIOTT, J., MACLURE, M. & SARLAND, C. (1996) Teachers as Researchers in the Context of Award

Bearing Courses and Research Degrees: final report to ESRC (Norwich, Centre for Applied Research in Education, University of East Anglia).

FENSTERMACHER, G. (1986) The philosophy of research on teaching: three Aspects, in: M. WITTROCK (Ed.) Handbook of Research on Education, 3rd edn (New York, Macmillan).

GIDDENS, A. (1984) The Constitution of Society: outline of a theory of structuration (Cambridge, Polity Press).

GIFFORD, B. & O'CONNOR, M. (Eds) (1992) Future Assessments: changing views of aptitude, achievement and instruction (Boston, Kluwer).

GIPPS, C. (1994) Beyond Testing (London, Falmer Press). GRAUE, M. E. (1993) Integrating theory and practice through instructional assessment, Educational

Assessment, 1, pp. 283-309. HAUSFATHER, S. (1997) A case of failed resocialisation? Action research and the struggle to redefine the

teacher identity, Educational Action Research, 5, pp. 363-382.

Developing Formative Assessment in the Classroom 631

HOLLINGSWORTH, S. (1997) Killing the angel in academe: feminist praxis in action research, Educational Action Research, 5, pp. 483-500.

KREISBERG, S. (1992) Transforming Power: domination, empowerment and education (New York, State

University of New York Press). MEHAN, H. (1979) Learning Lessons: social organisation in the classroom (Cambridge, MA, Harvard

University Press). NEWMAN, D., GRIFFIN, P. & COLE, M. (1989) The Construction Zone: working for cognitive change in

school (Cambridge, Cambridge University Press). O'HANLON, C. (1994) Reflection and action in research: is there a moral responsibility to act?

Educational Action Research, 2, pp. 281-290. TORRANCE, H. & PRYOR, J. (1998) Investigating Formative Assessment: teaching, learning and assess-

ment in the classroom (Philadelphia, PA Open University Press). TORRANCE, H. & PRYOR, J. (1999) Investigating and developing Formative Teacher Assessment in

Primary Schools, Final Report of Research Grant No. R000236860 to ESRC (Swindon, Economic and Social Research Council).

URDAN, T. & MAEHR, M. (1995) Beyond a two-goal theory of motivation and achievement: a case for social goals, Review of Educational Research, 65, pp. 213-243.

VYGOTSKY, L. (1986) Thought and Language (Cambridge, MA, Harvard University Press). WILSON, S. (1995) Not tension but intention: a response to Wong's analysis of the researcher/teacher,

Educational Researcher 24(8), pp. 19-22. WINTER, R. (1998) Finding a voice-thinking with others: a conception of action research, Educational

Action Research, 6, pp. 53-68.