R Hipkins9-10.10.08

Rosemary HipkinsNew Zealand Council for Educational Research

Assessment issues associated with Assessment issues associated with the implementation of NZC the implementation of NZC

Presentation to U Learn Conference, Christchurch, 9-10 October

R Hipkins9-10.10.08

The spirit of change in NZCThe spirit of change in NZC

• Learning how to learn – developing an identity as a ‘lifelong learner’

• School-based curriculum design is more explicit and linked to NAGs.

• A more holistic approach – interconnected nature of knowledge – coherence as a principle

• A more participatory view of learning and teaching – using learning, not just “getting” it

• Teaching as inquiry into our practice

2007

cf. 1993

R Hipkins9-10.10.08

Three key questions for this sessionThree key questions for this session

• What are the assessment implications if this spirit is to be enacted?

• Where might “twenty-first century” learning fit in?

• What part should the key competencies play in all this?

R Hipkins9-10.10.08

What does a learning to learn focus entail?What does a learning to learn focus entail?

• Learning how to learn is not a curriculum add-on. (Research suggests you can’t assess it as a thing-in-itself.)

• It occurs in the context of learning about something else but it won’t happen by osmosis. How you teach matters.

• There is an important dispositional component – you can’t force a student to be a learner.

• With support, students participate in making assessment decisions.

R Hipkins9-10.10.08

Key competencies

• Using language, symbols and texts

• Thinking

• Participating and contributing

• Managing self

• Relating to others

Lifelong learners (vision)

• Literate and numerate

• Critical and creative thinkers

• Active seekers, users, and creators of knowledge

• Informed decision makers

Learning to learn (a principle)

R Hipkins9-10.10.08

Creating a Creating a coherentcoherent curriculum curriculum

Lifelong learners

+

Key competencies

+

Learning area statement(s)

+

Selected AOs ICT

What are the assessment implications if we weave a more coherent curriculum like this?

Think of 3 possible labels for the weaving arrows….

R Hipkins9-10.10.08

My list of weaving possibilities …My list of weaving possibilities …

• Use of ICTs to foreground an aspect of learning and/or knowledge construction, not just information retrieval

• Future-focused themes (enterprise – E4E, sustainability – E4S, globalisation, citizenship)

• Inquiry learning, integrated learning, action competence projects

• Assessment methods that support learning conversations (NB aspects of this in PD for all of literacy, numeracy, AtoL)

• Values clarification activities, issues-based explorations

• Extra-curricular activities, school leadership roles and responsibilities

R Hipkins9-10.10.08

Weaving needs a warp and a weft – learning activities can help build knowledge into more coherent patterns but they are not a substitute for it!

R Hipkins9-10.10.08

Weaving new patterns for a new century?

Filling up with received knowledge (loosely woven)

Assumption that intellectual skills will be learned as content is mastered

Using knowledge in ways that strengthen the connections made

Explicit focus on scaffolding development of intellectual skills as well as content

R Hipkins9-10.10.08

Weaving new patterns continued

Students treated as “independent scholars” – knowledge is out there to be acquired ready made

Analytic thinking privileged

Focus on knowledge construction (the nature of subjects is important) and on learning in social as well as individual settings

Both analytic and creative thinking needed

R Hipkins9-10.10.08

Let’s practice more weavingLet’s practice more weaving

• Strategic awareness

More effective learners know more about their own learning.

They are interested in becoming more knowledgeable and more aware of themselves as learners.

They like to try out different approaches to learning to see what happens.

They are more reflective and better at self evaluation.

• Meaning making

Effective learners are on the look out for links between what they are learning and what they already know.

They like to learn about what matters to them.

These are two of seven factors in Bristol University’s Effective Lifelong Learning Inventory ELLI

R Hipkins9-10.10.08

Potential assessment foci in these factorsPotential assessment foci in these factors

1. Growth in knowledge/skills

2. New links made to existing knowledge

3. Evidence of a change of perspective

4. Reflection on learning strategies used

5. Engagement!

How many of these require a dimension of self assessment?

What support will teachers/student teachers need if what is assessed is to become much broader?

R Hipkins9-10.10.08

1. It takes careful research to clearly describe growth in learning in key areas

R Hipkins9-10.10.08

2. New links to existing knowledge2. New links to existing knowledge

• Essays that integrate across topics

• Concept mapping and other visual organisers

• Learning stories

• E portfolios

• Self reflection/learning logs

Who makes the judgement about the connections made?

What level of inference is involved in deciding how well/how many new links have been made?

R Hipkins9-10.10.08

3. Alfriston College students reflect on their 3. Alfriston College students reflect on their learning during a 3-day episode learning during a 3-day episode

After visiting Eco village, I compared their lifestyle to mine and ever since, I

have begun thinking differently

It is really important that New Zealanders especially Pacific Islanders are self

aware about diabetes and the ways they can prevent it from occurring even with just little lifestyle changes Just be proud of

who you are whether you're

Samoan or Tongan and getting to

know other cultures

I made beanies and booties for premature babies, this made me think differently as it's such a great cause to be helping and I

want to help others with knitting. Also I just want to volunteer for other causes

because it made me feel good to be helping someone else

R Hipkins9-10.10.08

Example 2: Here it is resilience

4. ELLI profiles could open an empowering conversation about being an effective learner

R Hipkins9-10.10.08

5. NZCER engagement survey provides one broad 5. NZCER engagement survey provides one broad snapshot to inform a complex issuesnapshot to inform a complex issue

The level at which students say they are “often bored in class”

R Hipkins9-10.10.08

Should we assess key competencies?Should we assess key competencies?

R Hipkins9-10.10.08

The “front end”

• Vision

• Values

• Principles

A potentially transformative package

The “back end”

8 levels

8 learning areas

8 sets of AOs per level

The revised package

What connects these two very different parts of the curriculum?

R Hipkins9-10.10.08

The “front end”

• Vision

• Values

• Principles

A potentially transformative package

The “back end”

8 levels

8 learning areas

8 sets of AOs per level

The revised package

What part do the “middle bits” (learning area statements, messages about pedagogy etc) play?

Might key competencies be the “glue” that brings all these pieces together?

R Hipkins9-10.10.08

Should we assess KCs?Should we assess KCs?

• If key competencies can transform learning outcomes, then asking how we might assess them per se seems to be the wrong question.

• Is this a better exploratory question: What can changed outcomes look like when key competencies are used to transform disciplinary learning?

We need lots of examples to “think with” as we explore this question

R Hipkins9-10.10.08

What do we do about “content”?What do we do about “content”?

Knowledge and its organisation

Teaching OF subjects

Based on Reid, 2006 – this fits comfortably with traditional curriculum planning models

R Hipkins9-10.10.08

Knowledge and its organisation

Capabilities

Teaching through knowledge FOR capabilities (i.e. key competencies)

Disciplinary knowledge is the basis through which we teach for capabilities (as outcomes in their own right)

KCs as the drivers of the spirit of the curriculum pose new assessment questions

What do these outcomes look like?What does progress look like?

R Hipkins9-10.10.08

Assessment and ULSTAssessment and ULST • We already know how to assess basic literacy and

numeracy well. Why would we change?

• If they are “necessary but not sufficient” to meet the multi-literacies demands of the 21st century, what other aspects of these learning foundations should we be looking to develop and assess?

R Hipkins9-10.10.08

Changes the key competencies can make are a bit like an iceberg

Transformative changes run very deep

The nearer the tip, the more obvious the potential changes

R Hipkins9-10.10.08

Where on the iceberg would you put these questions Where on the iceberg would you put these questions to ask while reading?to ask while reading?

• Is this text well written?

• Which voices, and interests are silent or absent from this text?

• What does the author believe is important?

• What are my assumptions as I read?

• Is the author attempting to manipulate me?

• What is the purpose of this particular style of writing?

R Hipkins9-10.10.08

Representing atomic structure Representing atomic structure

20th century (based on a true story: Rachel Bolstad circa 1991-1995):

21st century

• Open textbook.

• Turn to page about atomic structure.

• See diagram of atom provided by textbook, or drawn on board by your teacher.

• Accept that this is what an atom looks like. Redraw this model in test to show that you understand

• Experience shock and disbelief the following year when told that this is not the “real” model.

• Wonder when the teacher is going to finally reveal the “truth” about what atoms look like

Type the word “atom” and search using Google Images

R Hipkins9-10.10.08

R Hipkins9-10.10.08

Supporting deeper learningSupporting deeper learning

Understanding Language, Symbols, and TextsMaking explicit that these are representations of phenomena – (not

the “true” picture)

Shows that there are common conventions for representing certain concepts

• Who came up with these conventions?

• How did they decide?

• What kinds of evidence led people to model things this way?

• How does human experience shape the metaphors and models we can imagine?

(e.g. models of atoms - models of the solar system)

R Hipkins9-10.10.08

How might progress be described?How might progress be described?

More certain outcomes

Outcomes more uncertain

Zooming in Zooming out

Standardised testsmeasure traditional academic outcomes

Making rich connections across ideas and contexts

Key competencies transform learning outcomes

After Carr, 2008

Using new skills and knowledge in unfamiliar and more demanding contexts

R Hipkins9-10.10.08

NCEA as a lifelong learning initiative?NCEA as a lifelong learning initiative?

• Acknowledging successful learning rather than sorting students on unsuccessful learning to ration further study opportunities.

• Recognising a wider range of types and levels of learning - academic and practical.

• Providing different types of assessment opportunities.

• Providing multiple chances to demonstrate learning.

But.. We can’t fulfill this potential unless and until we align NCEA and the curriculum.

R Hipkins9-10.10.08

In my opinionIn my opinion• There should be strong links between the NZC and

the NCEA at the senior secondary level

• Assessment for qualifications should provide achievement information related to the purposes for learning that we say we value

• Such purposes should reflect current thinking about learning challenges in the 21st century

• The key competencies could provide a means of achieving this alignment

R Hipkins9-10.10.08

Pseudo ‘qualitative’ differences?Pseudo ‘qualitative’ differences?

Achieve: Knows some stuff

Merit: Knows quite a lot of stuff

Excellence: Knows heaps of stuff and some of it is really hard!

This type of thinking may be a misrepresentation of the intent of NCEA but it is not uncommon and too easily translates into negative and seemingly capricious assessment experiences for students…

It also leads to a ballooning curriculum

R Hipkins9-10.10.08

Rethinking A/M/E differencesRethinking A/M/E differences

• What sorts of things might students be expected to do with their knowledge if capabilities are the expected outcome from a learning area?

• Could you describe qualitative differences in how well they might do these things?

• If the answer is ‘no you can’t’ (i.e. the standard is basically about knowing stuff, or not knowing it) should this be assessed at ‘achieve’ level only?

R Hipkins9-10.10.08

An example from EnglishAn example from English

Reading and interpreting a poem by Robert Frost:• recognising the literal specifics of the text (on the lines)

• relating significance of text to own everyday experience (e.g. Maria sees the ‘watchman’ as a cop on his daily routine)

• Seeing correlates to more universal emotions and themes (e.g. Mary sees the clock as a symbol of the time Frost has left, telling him he can’t die yet)

summarised from Gee, 2000

There is a clear qualitative difference between each type of reading. What are the implications for:Exemplars? A/M/E criteria? Making reliable judgements?

R Hipkins9-10.10.08

The “take home” messagesThe “take home” messages

• Greater student involvement in all aspects of assessment is important if we really want to foster lifelong learning

• Where we already have good tools we should continue to use these well – again involving students in assessment conversations where possible (as in AtoL approaches)

• If key competencies do their work by transforming the ways we teach “content” asking if we should assess them is the wrong question

• However… we do need to find creative ways to assess the new types of outcomes that can be achieved when key competencies are positioned as integral to learning

R Hipkins9-10.10.08

ReferencesReferences

• Carr, M. (2008, 13 March ). Zooming in and Zooming Out: Challenges and choices in discussions about making progress. Paper presented at the NZCER Conference: Making Progress - Measuring Progress, Wellington, March 2008.

• Gee, J. (2000). Discourse and sociocultural studies in reading. Reading on-line www.readingonline.org/articles/handbook/gee/index.html

• Hipkins, R. (2007). Assessing key competencies: why would we? How could we? Wellington: Ministry of Education.

• Reid (2006) Key competencies: a new way forward or more of the same? Curriculum Matters, 2, 43-62.