Assessment in harmony with our understanding of learning

Mary James University of Cambridge, UKBERA President

Key questions

Assessments need to be congruent with our views of learning if they are to be valid

Questions for teachers: To what extent do the assessments required of me, available to me, or that I create, reflect the kind of learning that I aim to promote in students?

If assessments lack congruence with learning, how can I try to bring them into closer harmony?

Three generations of assessment practice

Three generations of ideas about links between assessment and learning and their implications.

Headings adapted from Chris Watkins’ (2003) descriptions of different views of learning: Learning is being taught Learning is individual sense-making Learning is building knowledge as part of doing things with others.

First generation: assessing learning of what is taught

individual performance under test conditions. ability to recall facts and demonstrate skills. Tests or tasks are specially constructed -

separate from learning. Preparation involves practice Time-limited No access to materials/resources Questions organised in a hierarchy of difficulty Responses assessed as correct or incorrect Comparison with other learners or against a

standard Areas for improvement inferred from incorrect

responses

Underpinnings:

Two different sets of assumptions underpin these assessment practices ‘Folk’ views of learning The brain or mind is a kind of vessel into which information is

poured. Learning is assumed to have occurred when knowledge is ‘retained’. Learner is passive Learning depends largely on innate ability (IQ).Behaviourist views of learning Learning as conditioned response to external stimuli Rewards and punishments ‘condition’ desired responses. Repetition of the stimulus makes the response habitual – value of

repeated practice. Eventually the response is so automatic that it can be produced at

speed. Complex skill broken down into components, taught separately, then

re-assembled. Economical to teach generic skills separately then applied in many

situations. Learn basic facts and skills first

Second generation: assessing learning as individual sense-making

Individual Problem-solving and understanding. Application of conceptual frameworks to find

solutions to problems, and demonstrate cognitive skills.

Specially constructed Tasks may be extended e.g. essays, open-ended

assignments, projects, coursework. Often time-limited - problem solving with ‘less

search’. Some access to materials - less a test of memory than

understanding. Assessed according to specified criteria - rarely one

correct answer. Normal trajectory of progress can be described. Improvement comes from closing the gap between novice

and expert. Areas for improvement inferred from misunderstandings

Underpinnings: Cognitive constructivist views of learning

Learning is determined by what goes on in people’s heads

Building mental models of how the world works helps people interpret new information

Active process of meaning-making. Emphasis on concepts Higher (or deeper) level processes such as

comprehension, application, analysis, synthesis and evaluation

Application of concepts valued for its capacity to reveal (mis)understanding

Novices need help to acquire the ways of thinking possessed by experts - the ways they organize knowledge and the way they control their mental processes through metacognition - thinking about thinking.

Third generation: assessing learning as building knowledge as part of doing things with others.

Not much evidence that this exists in schools

Underpinnings: The socio-cultural or ‘situated’ view of learning Learning involves both thought and action in context Thinking is conducted through actions that alter the

situation and the situation changes the thinking - the two interact

Learning is a mediated activity - tools and artefacts have a crucial role e.g. books, equipment, language and sign systems

Learning is a social and collaborative activity - people develop their thinking together

Learning is distributed within the social group e.g. language skill is not solely an indication of individual intelligence but the intelligence of the community that developed it

The collective knowledge of the group is internalised by the individual. As an individual creates new knowledge, then he or she will externalise it in communicating it to others who will put it to use and then internalise it (an expansive learning cycle).

Implications for teaching

Teachers need to create environments in which people can be stimulated to think and act in authentic tasks

Access to tools is important Activities that a learner can complete with

assistance so that the ‘more expert other’ can ‘scaffold’ learning.

Tasks need to be collaborative Students need to be involved in the generation

of problems as well as solutions. Teachers and students become a learning

community and jointly solve problems Most valued outcome is engaged participation in

ways that others find beneficial Problematic to judge an individual as having

acquired knowledge abstracted from action settings

Implications for assessment

Pointers for third generation assessment: Assessment alongside learning, not as an ‘after

learning’ event Done by the community - role for self-assessment, peer-

assessment and teacher assessment. Assessment of group learning as well as the learning of

the individual ‘In vivo’ studies of complex, situated problem-solving

i.e. participation in authentic (real-world) activities or projects

Focus on how well people exercise ‘agency’ in their use of the resources – proper justification for course-work assignments

Achievement captured and reported through narrative accounts and audio- and visual media

Portfolio has the potential to capture ‘assessment as inquiry’.

Holistic and qualitative judgement - not atomised and quantified as in measurement approaches.

Example 1: The ‘exhibition’ in US Coalition of Essential Schools (Ted Sizer, Grant Wiggins)

The exhibition brings together a number of dimensions of learning and meaningful assessment.  

It asks ‘students-as-workers’ to work across disciplines in a respectful way by creating ‘real’ learning activities.

Tasks are not necessarily devised by teachers; students can devise them for themselves, providing they understand the principles that underlie their construction.

It asks students to apply accumulated knowledge to new situations.

It insists on effective communication in a number of forms: oral, written and graphic.

It requires students to be reflective, persistent and well organised.

It creates a focus for learning by describing the destination for their journey, although precise learning objectives are not tightly pre-specified.

An example of a final performance across the disciplines

“Discuss behavior patterns as reflected in the insect world, in animals, in human beings, and in literature. Be sure to include references to your course work over the term in Inquiry and Expression, Literature and the Arts, Social Studies, and Science. This may include Macbeth, the drug prevention and communication workshop, Stephen Crane's poetry, ‘A Modest Proposal’ and other essays you have studied, Mark Twain's fiction, and behaviors you have observed in our School-within-a-School. You may also add references to what you have read about in the news recently.” (Melinda Nickle, Arkansas)

Procedure

Day one of the exam: You will be given four periods in which to brainstorm, make an outline, write a rough draft, and write a final copy in standard composition form. You will be graded not only on how well you assimilate the material but also how well you reflect the ‘student as worker’ metaphor and how responsibly you act during the testing period.

  Day two of the exam: You will assemble in villages of

three, evaluate anonymous papers according to a set of criteria, and come to a consensus about a grade. Each paper will be evaluated by at least two groups and two instructors. Again, a part of your overall semester grade will have to do with how responsibly you act and how well you demonstrate the ‘student as worker’ metaphor.

Example 2: UK Yr2 project using Mantle of the Expert (Dorothy Heathcote)

“Children and teachers work together to create an imaginary community within which they function as if they were experts e.g. mountain rescuers or archeologists. As the work progresses many possibilities begin to emerge which the learning community uses to define and deepen the imaginary world and explore the lives of the people that inhabit it. The community engages in a series of collaborative tasks, often motivated by a client’s demands, with teamwork, communication and problem solving central to the process. There is a group responsibility for the project as it progresses and the children act and make decisions with responsibility and authority, tackling authentic issues that seem purposeful and urgent to them.” (Jenny Lewis, Norfolk)

Range of assessment practices used

Ongoing dialogue Blog diary (http://theseacompany.blogspot.com/ ) Ongoing collective portfolio Individual learning diaries Daily sessions on meta-learning (Sniffles the Hopeless

Hamster) Self/peer assessment tools Home/school contact books Learning surgeries Connnections sessions Questionnaires (parents and children) Evidence of children’s work PSHE assessments e.g. Blob Tree; Feelings Wall Mind Maps Peer Teaching

Responses: Children

‘All our little ideas have made one big, fantastic idea.’

‘It’s like real life – there’s big trouble! We can learn about solving problems.’

 ‘It’s exciting because anything can happen.’ ‘It’s fascinating. You get to learn about things much

more. You’re learning things while you’re having big adventures.’

 ‘You can pretend to do things you can’t do in real life.’

‘I think it’s brilliant because there are so many dilemmas and adventures that never end. I learnt about myself that I keep trying until I get it perfect.’

‘I like it because it seems so real. I’m learning about other people and how to work as a team.’

Responses: parents

‘He will talk about what he has learned without prompting which isn’t normal.’

‘She’s definitely motivated by this type of learning – there’s obviously a great deal of enthusiasm for this project, not to mention the excitement factor. She is always pleased to tell us she will be doing company work today. It’s almost role reversal and she’s become the adult!’

‘There are lots of exciting things to do – we’ve heard all about who is doing what in the company, about diving – looking at the real gear - and about the history of the Titanic. We’ve taped TV documentaries and looked up things in books to feed the interest developed.’

‘It seems to give school work a purpose.’

Pros, cons and solutions

Pro: attractive to employers who claim they are interested in recruits who can demonstrate their capability to work in teams to find creative solutions to complex problems.

Con: raises questions about how to ensure the trustworthiness of such assessments when large numbers of students are involved and when those who are interested in the outcomes of such learning cannot participate in the activities that generate them.

Solution? The apprenticeship model with its concept of the guild as the guardian and arbiter of developing standards.

Another Con: within vocational education, systems of internal and external assessors and verifiers have attempted this although large-scale systems become bureaucratic, unwieldy and reductive.

Clearly, more work needs to be done

Is synthesis possible?

The big question is whether these theories of learning and their assessment counterparts are mutually exclusive, or whether, for practical purposes, they can be combined.

The teacher in Example 2 thought some combination is possible and probably necessary.

How does this square with current developments at national level in England?

Recommendations of the NCR Expert Panel, of which I was a member: a political compromise – or fudge?

N.B. The EP’s remit was to advise on ATs and issues of progression but not on the whole of the assessment system.

The Form of Programmes of Study and Attainment Targets

We emphasised the importance of establishing a very clear relationship between ‘that which is to be learned’ and all assessment.

We did not believe that Attainment Targets in the present level descriptor form should be retained. Instead, we suggested that the Programme of Study is stated as a discursive statement of purposes, anticipated progression and interconnection within the knowledge to be acquired. Attainment Targets should then be statements of learning outcomes related to essential knowledge (broadly conceived as facts, concepts, principles and ‘fundamental operations’).

Assessment, Reporting and Pupil Progression 1

We were concerned by the ways in which England’s current assessment system encourages a process of differentiating learners through the award of ‘levels’.

A distinctive feature of some of the high-performing systems is a focus on fewer things in greater depth in primary education, with particular attention on ensuring all pupils have an adequate understanding of key elements prior to moving to the next body of content.

Assessment, Reporting and Pupil Progression 2

We recommended an approach to pupil progression that emphasises ‘high expectations for all’.

The focus should be on ensuring all pupils are ‘ready to progress’ at the end of each key stage, having mastered the knowledge identified in relevant schemes of work and/or Programmes of Study.

This approach conveys necessary teacher commitment to both aspiration and inclusion, and implies the specific set of fundamental achievements that all pupils should attain.

What have Ministers decided to date?

To abolish NC levels To retain some form of KS2 testing in Ma and

En (Bew Report) To ‘reform’ 16+ qualification by:

Replacing GCSEs with EBacc Certificates (EBCs) in academic subjects

Getting rid of modules, coursework, controlled assessments, resits etc

Introducing 3 hour exams and making them harder, with no access to resource material

Replacing A*-G grades with Grades 1-6

Is all lost?

Not necessarily It is reasonable to expect public demand for some kind

of external summative assessment in basic skills at the end of KS2 and in the subjects of a broader curriculum at the end of KS4

But, if Government is genuine in its encouragement of more teacher and school autonomy, there is space for curriculum and assessment innovation that might itself have a beneficial washback effect on achievement - and the nature of terminal assessments in the long term.

But this will depend on teachers having the confidence to seize the initiative. They are unlikely to be able to do this alone. They need partnerships.

Remember that the average term of an Education SoS is less than two years, and a Government less than 5 years.

Conclusion

Overarching principles are ‘fitness for purpose’ and ‘validity’ - assess what you value rather than value what you assess.

Assessment washback is powerful, especially in high stakes settings, so it is imperative to achieve a better alignment between assessment, teaching and learning.

There may be possibilities for synthesis whereby a more complete theory can emerge. The possibility for a more complete and inclusive theory of learning to guide practice of teaching and assessment seems to be a goal worth pursuing.

References

James, M. (2006) Assessment, teaching and theories of learning. In J. Gardner, Assessment and Learning, London, Sage.

James, M. (2008) Assessment and learning. In S. Swaffield, Unlocking Assessment, London, David Fulton.

James. M. with J. Lewis (2012) Assessment in Harmony with our understanding of learning: problems and possibilities. In J. Gardner, Assessment and Learning, Second Edition. London, Sage.