ASSESSMENT OF INTERCULTURAL LEARNING

Introduction to the project:

In 2007 and 2008, I was part of a project conducted by the Research

Centre for Languages and Cultural Education at the University of South

Australia, under the leadership of Angela Scarino and Tony Liddicoat. The

project was on Assessing Intercultural Learning, funded by the Australian

Research Council. I was one of three practising teachers from the School of

Languages in Adelaide involved in this project along with about 6 other

teachers from a number of primary and secondary schools across

Adelaide, representing the government, Catholic and Independent sectors.

I wish to stress that any views and conclusions related by me today are

not necessarily those of the researchers. They will no doubt have drawn

their own conclusions. These comments are merely personal observations

as a participant in that ARC project although I will draw on some thinking

and considerations from the research team as we discussed the

challenges and the concepts of assessment as a group throughout the two

phases of the investigation. It became evident very quickly to the team

and participants involved how complex this aspect of teaching and

learning in the classroom actually is as far as intercultural learning is

concerned.

The Intercultural Language Teaching and Learning in Practice (ILTLP)

Project mobilised many teachers of languages into considering the

development of intercultural programs in their classrooms, to view

language learning and teaching as a relationship between language and

culture. It has required teachers and students to consider or re-evaluate

their own cultural positions with respect to the linguistic and cultural

material with which they work.

Adopting an intercultural orientation, however, has also required us to

consider the assessment of an intercultural capability on the basis that

assessment and learning are inextricably linked. Specifically, we have

1

needed to consider what indicators of intercultural learning might look like

and how we might assess such learning. We needed to consider what

evidence to collect, and in a practical sense, what can be collected in the

classroom, in order to make any valid judgements of intercultural learning.

Since the teachers involved in this project represented such varied levels

of teaching in the classroom, challenging questions were raised about the

kinds of intercultural learning that might take place in a year 2 or 3 class

as opposed to a middle school classroom or yet again, in a senior school

classroom. Are there different assessment procedures that would be

appropriate for different cohorts?

The Process

As participants in the ILTLP project will know, everyone had first to reflect

on his or her own cultural position, to clarify cultural identity and then to

conceptualise what the intercultural meant. Broadly speaking, and to state

the now fairly obvious, participants understood intercultural teaching and

learning to be developing an awareness and understanding of linguistic

and cultural diversity, of seeing themselves and their students as both

language users and as cultural beings. The intercultural works from the

premise that both language and culture affects communication and

relationships, and that an awareness of behaviour and attitudes towards

language and culture, of the multiple interpretations in interaction that are

possible, and of the nature of the resource materials we select in the

classroom, are very important considerations.

The first challenge I and others in the team experienced in this project was

something of the obvious - to decide exactly what we wanted students to

learn conceptually, and whether the planned interactions were going to

elicit the sought after evidence of intercultural learning. In other words,

what kinds of formative and summative assessment tasks were going to

be useful in determining whether intercultural learning had taken place?

2

It is worthwhile highlighting the diversity of views held by participants in

this project. During initial discussions in the first round, some believed that

it was simply not possible to assess intercultural learning in conventional

ways, arguing that a student’s development of the intercultural is very

much tied up with the growth of the individual, the socio-cultural

development of the student, if you will. This growth is organic, continuous,

and unpredictable, manifesting at different stages for different students. It

is a development linked to the whole notion of learning, and even learning

for learning, which could only be arbitrarily “assessed” for something like

intercultural understanding.

On the other hand, given that all learning is along a developmental

continuum, even though not always predictably linear, it seems not out of

the question to be able to take a snapshot along this continuum, to

examine tasks or performances which can conceivably provide evidence of

intercultural learning.

Increasingly, we now find that writers of curricula at systems levels are

now including the notion of the intercultural and, rightly or wrongly,

believe that it is a capacity that can be ticked off as a performance

standard within the context of conventional approaches to assessment

tasks, much in the same way as higher order thinking or treatment of a

topic in depth might be able to be ticked off. The tension between these

two polarities, that is, the views that the intercultural is assessable or not

assessable, certainly makes this area of investigation challenging, not the

least from a practical point of view of the daily teaching routines within the

classroom.

It is not possible to give you a detailed description of the kinds of units of

learning planned by the teachers of Italian, Spanish, Chinese and French

involved. However, in the first phase of the research, there were “topics”

for want of a better description that ranged from asking year 2 students of

Italian to deal with the formalities of greetings, and what it is like to live in

3

Rome for Year 10s. There were excursions into the culture of the

indigenous of Mexico in a Beginners Spanish class which included adult

learners, contrasting these with the treatment of the indigenous in this

country. Year 9 students of Chinese looked at schooling while the role of

women in China was examined by a class of year 11s. All the while,

teachers aimed to get students to be clear on their cultural position while

examining the culture of others. This was asking students to demonstrate

a sophisticated skill.

The French teachers organised activities which included looking at the

Parisian Banlieue in a year 12 class, the Pari-roller event in the context of

Youth Culture in a year 10/11 class, and the nature of lunch in Australia

contrasted with La Cantine in a French école primaire for a group of

primary year 5 and 6 group.

If I can highlight Kathy Moore’s endeavour, the primary school French

colleague with a group of year 6 primary school students who only have

50 or so minutes per week of French, who focused on La Cantine. The

purpose of the assessment was for students to recognize:

that there is not one way to have lunch and

that ‘lunchtime’ means different things to different people

There was an initial discussion in English about the students’

understanding of what a canteen is, comparing experiences from their

primary school and other schools within Australia and overseas.

Interactive tasks included:

An initial discussion in English about their understanding of what is a

canteen, comparing experiences from Glen Osmond Primary and

other schools within Australia and overseas

A written task in English responding to: Describe how you imagine

la cantine in a French school to be.

Spoken responses in French of meal preferences in a French cantine.

4

Written examples in French of meal preferences taken from a French

school menu

Written self-reflections about lunchtime in France and at Glen

Osmond Primary School, after working with several French texts.

Students then had to write reflections to the following questions:

1. Imagine you are going to spend a week or so with a family in France.

You will be going to school with them. What will it be like at

lunchtime?

2. A French student is visiting Glen Osmond for a short time and will be

with your class. What do you need to tell them about lunchtime in

your school in order for them to feel comfortable?

3. Where would you prefer to spend your lunchtime? Explain your

reasons.

The latter tasks clearly required students to apply the knowledge gained of

the other culture and put themselves in a position in which they had to be

aware of their own cultural practices and values. Kathy reported several

instances where some students had an “ah ha” moment, as they came to

realise such marked differences between French and Australian lunch

routines, as well as pondering the origins of word used.

I started the first round of trialling taking the perhaps naïve viewpoint that

the intercultural can be gathered principally in the traditional form of a

summative task, one I had to consider undertaking, given the time

constraints at my disposal in the course. My aim was to:

To assess the students’ writing ability in French as a response to a

written text (a SACE required task).

To assess the students’ understanding of the cultural context of the

text to which they must respond.

To assess the students’ appreciation of their own cultural

perspective in writing the response.

5

After a series of interactive and comprehension activities, and reflecting

on what students had gained from these, the final task was for students to

write a response in French to a letter by a citizen complaining about

roller-bladers riding on the footpaths, endangering others and preventing

the complainant, in particular, from walking his dog in peace. (This

complaint was authentic as it was also listed on the Pari-roller website.)

Students had to examine this complaint as an Australian outsider, being

mindful of their own cultural position and that of French participants. This

was an explicit criterion. Not all students were able to manage the task in

the way I had hoped.

The re-assessment

Now, you can see in these activities that they are practicable enough in

the classroom, of reasonable interest and authenticity to students, and

forming the basis of plenty of language learning regardless of whether or

not anything specifically of intercultural value could emerge from them.

And it was precisely this fact that was the basis of a whole raft of

questions that were raised in the evaluation of the first round of the trial.

It is a somewhat sobering experience to be grilled on why we did the

things we did? Did we elicit the kinds of evidence we were expecting? If

not, why not? What did we actually want students to learn from an

intercultural perspective?

While some of my students were able to respond in articulate and

interesting ways, there were others who could not interpret the task as

required, even after one-on-one discussions. I had several students from

Africa in the group who were finding adjusting to Australian ways of

learning challenging enough! A Chinese teacher who looked at the role of

women in Chinese society with a group of year 11 girls also had some

sophisticated responses from the students. However, she, like many of us,

found that she had to revisit the notion of the intercultural and ask herself

6

whether students could actually go beyond the obvious comparison

between cultures and sometimes sophisticated statement of opinion from

a linguistic point of view. Where was the evidence that students were

able to interpret the differences and similarities and take an

intercultural position?

Many participants also found that there was a greater amount of time

spent discussing ideas in English with the students rather than in the

target language in order to get across what the students should aim to

achieve. In other words, we found that classroom time readily focused on

the culture and rather than the language, or conversely, focused on the

language at the expense of the culture. Here entered a tricky balancing

act.

In evaluating what had occurred during the trial and after the first round,

the researchers were formulating the view that the traditional ways of

measuring achievement, as I had certainly adopted, were insufficient for

assessing the intercultural. The implication for us was to shift from the

tasks created for the class to emphasising what have been described as

“data points”, that is, from an emphasis from stand alone procedures, or

structured activities (tests or tasks) to the totality of formal and informal

judgements made in interaction.

As participants, we were asked to submit a range of these data points,

along with a description of the interaction, what we hoped would be

achieved in each interaction, and the evidence that arose out of it.

Interactions could be written tasks in [language] or English, recordings of

conversations and interactions, formative assessment snapshots etc.

A common observation made by several participants was the following:

The trouble is that sometimes you get an insight from a student

through spontaneous and unplanned conversation. It is difficult to

formally call that a data point for collecting evidence, at least, at a

7

practical level in the classroom, even though, in a sense, it is

actually a data point at which some “evidence” emerges. How to

practically capture them is the challenge.

The second round

In other words, going into the second round of trials, we were looking at

assessment from a different perspective. Assessment was viewed more as

something socio-cultural, interactive and as activity rather than as a single

event, procedure, task or episode. Assessment became the developmental

path and the inquiry for both teachers and learners. And as assessment

was reconceptualized, the process of eliciting evidence, validating and

judging it changed.

In the literature, assessment is described as being of three kinds, broadly

speaking. There are the inherent assessments, that is, those that happen

informally and non-verbally in all situations. I have already mentioned that

many commented students made off the cuff remarks that actually

demonstrated an intercultural understanding but which were not captured

in any formal way although teachers wished they had been able to do so.

There are also the discursive assessments, that is, those that occur when

members of a social group talk about what they are doing in an evaluative

way. Lastly there are the documentary assessments that occur when

activities are evaluated according to a scheme that produces numbers and

symbols (in other words the formative and summative assessments with

which we are familiar).

To reorientate our thinking, Angela Scarino cited a quotation from

Delandshere, 2002, that read:

“We are moving from an educational practice of assessment where we

have defined a priori what we are looking for, to an educational practice

where we are participating in activities in which we formulate

8

representations to better understand and transform the world around us. If

our purpose is to understand and support learning and knowing and to

make inferences about these phenomena, then it seems that the ideas of

inquiry – open, critical, dialogic – rather than assessment (as is currently

understood) would be more helpful.”

Our second round of trialling assessment, therefore, was mindful of the

idea that one experience, whether a text, a DVD, an image and so on, is

just one experience and will never provide a complete picture. For

students to decentre, that is to see culture and language from the inside

and the outside, is the more critical factor. The data we intended to collect

would need to be recordings of interactions that happened from moment

to moment (an extremely tricky procedure), surveys and interviews and

other forms of self-reporting, and also our own recordings of the things we

had observed happening in the classroom. Of course, samples of written or

spoken tasks would also be a feature.

In the assessment process, we were continuing to move forward with key

questions that were posed from the outset of this research project:

How do we move from the cultural to the intercultural?

What exactly do we want students to learn? What language is

needed to do so?

Why is it important?

What data or evidence captures the learning?

What are the questions we need to ask students to progress to the

intercultural?

What processes will we use to gather this? How can we connect the

“data” points?

To capture the data, the focus was more on the assessment than the

content, on how students were developing. In my own class, I certainly

found it necessary to focus more on the progress of a small group of

individuals rather than the whole class of 26 students. I have had to

interrogate more vigorously the relationship between language and

9

culture and to interrogate the students about this connection. For

example, given the text, why is the structure before you the way it is, why

is this or that vocabulary or language used, what is not said?

Rather than focus on the end product, the focus had fallen more on the

process of thinking in getting there. We were looking more for cultural

positioning from students and asking students to respond as themselves

rather than as an imaginary person or in an imaginary situation. (Exams

are full of questions that ask students to imagine themselves to be this or

that person or to be in this or that situation rather than responding to their

own feelings, values and knowledge directly.)

In the second phase of research, I had a class composed of a different mix

of students: 4 year 10 students (I boy and 3 girls), 23 year 11 students (10

males and 13 females). Of the 10 males, there were 6 Africans boys, 1

German, 1 Laotian and 2 Anglo-Australians. Of 13 females, there were 3

German girls, 2 Africans, 4 Anglo-Australian girls and 4 Anglo-Australian

adults. Given the size of the class I decided to focus 10 of these students

for collecting data and representing a cross section of male and female

students and their backgrounds.

The topic we studied was multiculturalism, broadly speaking, with some

emphasis of the prohibition of the wearing of the hijab in French schools.

There were 9 major activities which included students keeping a journal

which they could write in English or French (some of the Africans found

French easier), recordings made of some interactive tasks, oral reports in

French on multiculturalism, comprehension exercises on various texts

including writing personal opinions, and a final written task symbolising a

cumulative end-point. All these activities served as data points for

evidence. There were also the required moments of explicit grammar

teaching and the introduction and revision of vocabulary.

10

Just to highlight two activities: the first and the most revealing was one in

which students were asked to answer a series of questions about the

culture they most identify with, and then to talk to a person in the class

they had not talked to share their answers. You can imagine the intensity

and amazement of Aussie students who learnt about Kenyan or Congolese

wedding ceremonies or attitudes, or adults talking to young Burundi boys

about Australian insights. And as often happens in the classroom, I was

slow enough not to realise that I had not adequately catered for recording

some of the interactions!! Nevertheless, this was a genuine intercultural

moment, so to speak, since students suddenly had insights into the other

and their own cultural headsets.

For the culminating activity, students were presented with two sets of

photos and had to choose one set, to write a response in French by

interpreting these pictures and their feelings, and try to express their

cultural point of view. Before this task, and in fact during the series of

activities, I had to explicitly remind students what we were aiming to

achieve, that is, to explain again what the intercultural actually entailed.

What students were asked to do was quite challenging, abstract in some

ways, and requiring them have an adult headset. One student only

actually analysed the language written across the photo of a Muslim girl

wearing the hijab. The girls wearing the hijab in the tricolor was better

understood in terms of the cultural context.

As a general observation of interaction in the class, I found that the African

students participated less publicly in whole class discussions while the

Anglo-Australian girls were the most vocal. The class as a whole was fully

engaged, however, and many expressed an appreciation of the

opportunity to deal with the topic in the manner that we did.

On the question of multiculturalism the African and German students

thought Australia was a multicultural society while the Anglo-Australian

11

students thought Australia was on the way of being one but that there was

still quite a bit of intolerance.

Being involved in this project has certainly changed my view of how I

teach and it is difficult not to see every resource used now in terms of a

cultural context and the potential for some intercultural understanding.

However, I do see on-going problems with assessing intercultural learning;

while curriculum designers increasingly frame the intercultural component

into formulaic assessment, I feel uncomfortable about it being something

easily accounted for in performance. The reality is that we are dealing with

a student’s development and internal state of cognition that is not easily

accounted for at a given moment. Only the most articulate, perhaps, will

be able to express their insight within a one off assessment piece.

I find it difficult to evaluate how much my explicit teaching of the teaching

of the intercultural actually determined the responses rather than there

being an unexpected and valuable “intercultural” revelation from the

students. Maybe it does not matter. I was certainly taken aback when one

of the students in a piece of writing, which was in the form of a dialogue,

had one of the characters remark: “Oh you are an interesting intercultural

person!” And so here we have a case of the concept already having

become what I call a commodity, something that is packaged, and in that

sense, potentially removed from authentic interaction.

Certainly, intercultural teaching and learning is not an easy recipe or

methodology that is a panacea for learning language and culture, and for

going beyond cultural comparisons. It requires a different kind of alertness

and different kind of questioning from the teachers to progress the

intercultural learning in the students. It will take some experience to

capture the right bits of evidence to show that education, or self-education

in a broad sense, has sparked young minds into looking at themselves and

others in an exciting way.

12

Of course, good teachers have always had elements of the intercultural in

the classroom. Good teachers have always managed to ask the right

questions to progress their students’ learning. Through the experience of

this research project, I hope to join them.

Joe van Dalen

Acting Deputy Principal

School of Languages

255 Torrens Road

West Croydon SA 5008

Joe.vandalen@schooloflanguages.sa.edu.au

Mob: 0421 954 216

July 10, 2009

13