ASSESSMENT AS LEARNING: ENGAGING STUDENTS IN ACADEMIC LITERACY IN THEIR FIRST SEMESTERTimothy Jowan Curnow & Anthony J. Liddicoat

Research Centre for Languages and Cultures

University of South Australia

REDESIGNING ASSESSMENT

Redesign applied linguistics courses, including the assessment

Integrate academic literacy of native English speakers and others into teaching, learning and assessment

Focus on academic literacy doesn’t entail a lack of focus on disciplinary content; the two can be integrated

A MODEL OF THE ASSESSMENT PROCESS

Conceptualising Judging ValidatingEliciting

What to assess

How to judge

How to justify

How to elicit

The “ability” or “knowing” of

interest

Criteria for judging

performance, including

1. aspects of competence

2. performance analysis

ValidationTasks/ procedures

that operationalise the construct

Source, Scarino, 2006)

ASSESSMENT IN PRACTICE

Most thought is given to elicitation, very little to anything else. In terms of this model, many assessment codes of

practice deal only with elicitation (and only with the amount of elicitation)

Note that a lack of clarity about the conceptualisation of what is being assessed has a direct impact on all other assessment processes

RETHINKING ASSESSMENT IN THE APPLIED LINGUISTICS MAJOR

Design of an overarching assessment approach for all core courses in applied linguistics

Process: Conceptualising what learnings we wanted to achieve

across the program Conceptualising how these interrelated Conceptualising what tasks would:

allow those learnings to be put into practice give evidence of nature of students’ learning develop students knowledge of the discipline as well as

demonstrating their ability to use this knowledge

Designing tasks (eliciting) Designing assessment criteria (judging)

WHAT LEARNING DID WE WANT TO SEE?

Not simply “body of knowledge” but creation/use of knowledge in different ways. E.g. acquiring knowledge from text linking argument and evidence evaluating information according to purpose ……..

Focus on transformation of knowledge not of reproduction of knowledge. E.g. using knowledge for new purposes, not simply recall making connections between different texts,

experiences, etc. ……

WHAT LEARNING DID WE WANT TO SEE?

Key dimensions determined by team as being1. Critical reading of research2. Analysis of research writing3. Synthesis of research from multiple sources4. Constructing an argument using the research of others5. Analysing language data6. Constructing an argument from language examples7. Understanding the process of research development8. Designing and implementing research projects9. Communicating research findings

Divided between courses in the major 1-4: LANG 1056 5-6: LANG 1055 7-9: Later courses

HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT?

Realisation that some assessment tasks did not have a clear link either to knowledge or its use, for example: Reading tasks

Purpose: “get the buggers to read” Reading had no purpose other than having been completed Questions mainly showed comprehension of the text (i.e. they

confirmed the text had been read) The tasks had no connection to each other or to larger themes

Oral presentations on texts Purpose: ??? (something oral might be good? Task “inherited”)

Presentations were boring, for tutors and for the students – simple rehearsals of information

Students had nothing to say about the texts because they had no reason to say anything

Students had nothing to say about the texts because they had no framework in which to develop any ideas about it

Very hard to mark (criteria when you don’t have a purpose?)

HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT?

New assessment plans have a reduced diversity of task types, at least for first year

courses greater coherence between individual assessment tasks

cumulative development across tasks development towards a clear end point, with each task

contributing to the end point

clear links between teaching/learning needs in class and assessment tasks

ASSESSMENT IN LANG 1056 ‘LANGUAGE AND CULTURE’

Developmental focus on essay writing and reading/using research writing

Academic literacy focus combined with content focus

Discrete literacy focus for each assessment task Cumulative learning developed through assessment

ASSESSMENT IN LANG 1056

Critical reading task guided reading task

based on a single set research article

emphasis on identifying and analysing arguments and evidence

emphasis on abstracting information across components of the research article

ASSESSMENT IN LANG 1056

Analytic reading task focused essay-like

response based on a single reading – using a research article to address a new question

essay topic related to reading but not about the reading – knowledge transformation

emphasis on locating and using information for new purposes

requires critical reading as starting point

ANALYTIC READING

Text:Wierzbicka , A. (1991) Cross-cultural pragmatics. The

Hague: Mouton de Gruyter. Ch 2: “Different cultures, different languages, different speech acts”

Task: Your task is to write an analytic response to

Wierzbicka’s paper which addresses the following topic:

The meaning of an utterance in a language is not simply the result of the meanings of the words in the utterance, but is fundamentally influenced by the cultural context in which it is uttered.

ASSESSMENT IN LANG 1056

Analytic reading task focused essay-like

response based on a single reading – using a research article to address a new question

essay topic related to reading but not about the reading – knowledge transformation

emphasis on locating and using information for new purposes

requires critical reading as starting point

ASSESSMENT IN LANG 1056

Synthetic reading task focused essay-like

response based on a set of readings

essay topic related to readings – knowledge transformation

emphasis on using multiple sources to address a question

emphasis on synthesising arguments and evidence from multiple sources

requires critical reading and analytic use of reading as starting point

SYNTHETIC TASK

Texts:Nelson, et al. (2002). Cross-Cultural Pragmatics: Strategy Use in

Egyptian Arabic and American English Refusals. Applied Linguistics 23(2):163-189.

Hickey, L. (2005). Politeness in Spain: Thanks but no “thanks”. In L. Hickey and M. Stewart (eds). Politeness in Europe. Clevedon: Multilingual Matters.

Haugh, M. (2003). Revisiting the conceptualisation of politeness in English and Japanese. Multilingua 23(1-2): 85-110.

Task: Using only these readings, answer the following question in a brief

essay format:

While all cultures value politeness, what it means to be polite varies from culture to culture.

ASSESSMENT IN LANG 1056

Synthetic reading task focused essay-like

response based on a set of readings

essay topic related to readings – knowledge transformation

emphasis on using multiple sources to address a question

emphasis on synthesising arguments and evidence from multiple sources

requires critical reading and analytic use of reading as starting point

ASSESSMENT IN LANG 1056

Essay conventional essay task requires critical reading

and analytic and synthetic use of readings as starting point

involves locating own sources

ASSESSMENT IN LANG 1055

Focus of course is on micro-analysis of language data

Academic literacy focus: analysing language data constructing an argument based on language examples

Previous assessment items included a ‘conventional’ essay, requiring finding own references

Research literature important and much exposure in course, but didn’t need assessment item since focus of LANG 1056

ASSESSMENT IN LANG 1055

Language analysis tasks One task per course module

semantics ‘grammar’ (morphology and syntax) phonetics and phonology

Modelled in tutorials spot patterns in the data develop rules to account for the patterns (choosing between competing analyses)

Assessment criteria include treating the data as the unit for analysis analysis justified on the basis of the data

ASSESSMENT IN LANG 1055

‘Data analysis and justification’ essay replaced previous ‘conventional’ essay half-way through course structure modelled on previous six or seven readings students choose one of eight languages (they’re nearly

all studying a language also, strong engagement) given 30-40 example sentences initial analysis, then an explicit choice, each path with

advantages and disadvantages students explicitly told there is no single right answer need to give evidence for their particular answer, they

are assessed on their justification, and how they argue that their analysis is superior to alternatives

CONCLUSIONS

Important to begin from what you want students to learn

‘Whole-of-major’ approach Students engaged with professional research

literature Academic literacy through discipline-specific

content Effect on students somewhat unclear, since most of

these students are only now finishing their first year, but some tentative outcomes

OUTCOMES?

Course evaluation for LANG 1056 Did the course make you feel more confident in dealing with

reading applied linguistics research? Resounding ‘yes’. Did the course develop your understanding of the types of

writing required in linguistics? Resounding ‘yes’. “I came into the course with little knowledge on reading

linguistics research and now I feel as if I can read research papers confidently”

“I went back over the first reading of the course and I discovered that I could actually understand what the article was trying to say, whereas before, I had much trouble understanding”

Students’ results LANG 1056 essays brought total marks up Some LANG 1055 essays publishable in terms of structure

and arguments