View
236
Download
4
Category
Preview:
Citation preview
Sample placement test_Jackie Cipollone_110316.doc Page 1 of 29
Sample placement test
Rationale for development
The purpose of the placement test is to help determine whether people
presenting for English language tuition for the first time are most
appropriately placed in a Certificate I level course or a Certificate II level
course. Each of the courses comprises groups of units of competency at two
outcome levels. This allows teachers to place students in either full-time or
part-time classes, and provides a flexible and scaffolded pathway to allow for
the learning needs of students who may require a slower pace of learning.
The minimum entry requirement for the Certificate I is proficiency
approximating International Second Language Proficiency Rating (ISLPR) 0,
and the minimum entry requirement for the Certificate II is ISLPR 1. The
outcomes of each course are also expressed in terms of the ISLPR. Course
content is customised according to the goals of the learners and usually has
one of three focuses: employment, further study or community.
The placement process is designed to enhance learning opportunities by
enrolling students in classes that will allow them to increase their overall
English proficiency in the context of their personal goals. In that sense it is a
low stakes test, as it is not intended to exclude students from receiving
tuition. In the event that students are incorrectly placed within one of the
two levels of each Certificate course, they can be transferred to another class
Sample placement test_Jackie Cipollone_110316.doc Page 2 of 29
within the first three weeks of the course beginning. However, it is
problematic if a student has been placed in the wrong Certificate level. There
is the administrative hassle of transferring the students to another course
and the ensuing frustration for already time-pressured teachers. In addition,
if a student enrols and is discouraged as a result of inappropriate class
placement s/he may drop out, which increases attrition rates.
A placement test, then, needs to be able to identify presenting students who
are operating at approximately ISLPR 1, and to distinguish them from those
who are operating at either above or below approximately ISLPR 1. If a
student is below ISLPR 1, it is safe to place them in the Certificate I course;
if at approximately ISLPR 1, then the Certificate II course can be considered.
If a student demonstrates as part of the placement process that s/he is
significantly above ISLPR 1, a Certificate III course can be considered
(minium entry requirement ISLPR 2). In the current vocational education
and training environment successful completion of a Certificate level course
will help secure further funding, so it is important to place students in the
level course that they have the best chance of successfully completing.
Placements tests are usually designed at a local level. Teachers –
particularly the part-time teachers who deliver programs in more than one
college - have reported that there is sometimes a problem of consistency
across colleges. This problem may be compounded if some teachers are
more familiar with the curriculum than others. Another issue is that
Sample placement test_Jackie Cipollone_110316.doc Page 3 of 29
placement tests tend to ask students to do things they have done many
times before in similar circumstances. For instance, students who are
coming to TAFE from another provider have often been asked before to write
about their life in Australia, or to talk about their employment in their home
country, and because of this they can demonstrate a false level of
proficiency. In addition some sections use tools such as cloze passages or
comprehension-type tests which, while quick and easy to mark, are oriented
towards knowledge about English rather than use of English and in that way
are inappropriate for determining placement in communicative classes.
This test, then, attempts to provide an easy to administer and mark tool to
help teachers place learners in the most appropriate groups that will allow
them to progress in their English language learning.
Description of the target group
People for whom the placement test is designed are adult men and women
from language background other than English who present for English
language tuition at TAFE NSW colleges for the first time. These people will
have a variety of linguistic, cultural and social backgrounds, but usually fall
into three broad groupings according to their English language acquisition
experiences. The first will be people who have had previous English training,
either in their home countries or in Australia (usually at AMES). The second
will have had no previous English language training, but may have acquired a
degree of English proficiency through working or living in Australia or another
Sample placement test_Jackie Cipollone_110316.doc Page 4 of 29
mainly English-speaking country (usually New Zealand). The third will have
minimal English proficiency. Increasingly, the people who fall into this third
grouping will have had little or no formal education in their own countries.
These new students will join other students who have already completed
some English training at TAFE. These ‘old students’ can be placed in the
most appropriate classes based on their previous course achievement and so
do not need to undertake the placement test.
The courses in which the students will enrol comprise speaking, listening,
reading and writing units of competency whose contents have been informed
by the significant features of language described in the ISLPR scales. Most
delivery and assessment of the units of competency is holistic across the
macroskills and the courses are designed to facilitate communicative
language teaching methodology. For delivery and for assessment of learner
achievement of course outcomes, the use of texts for the learners’ real life
purposes is stressed and these are required to be authentic in that sense.
However, there is a recognition that texts need to be appropriate for the level
at which learners are currently aiming, and this may require some
modification of real world texts.
The item/task development process
The starting point was a review of the relevant literature, and the following
theoretical principles informed development of the tasks.
Sample placement test_Jackie Cipollone_110316.doc Page 5 of 29
Assessment as interaction The tasks are designed to stimulate authentic
engagement on the part of the students. The test doesn’t look like a test a
much as it does a gathering of useful information that will benefit the
student. For instance, question 2 in the written component asks students to
indicate their future goals, and question 3 requires some self-assessment on
the part of the student. The interview component is an interpersonal
interaction for an authentic purpose, as the preferred enrolment and child
care or parking requirement items provide information that is genuinely
needed. The map reading and return date items have also been incorporated
into the test for the purpose of student engagement. Spence-Brown notes
that “authentic engagement with the task as interaction … may lead to higher
validity” (2001: 479).
Communicative competence The placement test is designed to provide
information that is needed, and is not a test without authentic purpose.
“What students are asked to do … approximates as closely as possible to
real-life language use” (Alderson et al., 1995:14); “we should make [test
tasks] mirror salient features of the real-life situation” (Weir, 1993:16); we
need to “select appropriate texts, to be read for realistic purposes” (Weir,
1997:39). The information gathered by the placement test is used to place
students and uses means typical of what they will encounter in their future
classrooms. The test states up front who the audience is, and the use to
which the information gathered will be put.
Brindley points out that listening tests in particular can be very stressful for
students (1998:176). The interview has been chosen because it is more
Sample placement test_Jackie Cipollone_110316.doc Page 6 of 29
‘natural’ than, for instance, a pre-recorded input text used as the basis of an
aural comprehension. It is hoped that this may make the testee feel less like
s/he is being assessed, particularly as the purpose for the information
gathering is authentic. The face-to-face aspect of the interview also provides
the visual support that Brindley refers to as one advantage of the use of
video (1998:180).
Criterion referencing The placement test is criterion referenced. Unlike
tests which place the top group of interviewees in the top class, the next in
the middle and the bottom in the lowest class (as cited by Alderson et al.,
1995:11), this test is criterion referenced against the ISLPR. Like the scale
itself and the courses that have been informed by it, the placement test
seeks to sample “language as it is used by learners in real life” (Wylie and
Ingram, 2007:iv).
Test for best The test is designed to allow students to demonstrate what
they can do, not what they cannot do – “We should always test for best”
(Weir, 1993:5). It is not meant to exclude students, but to place them
appropriately in order to maximise their potential for success.
Direct testing Speaking and listening are assessed by asking students to
speak and listen; reading and writing are assessed by asking students to
read and write. The tasks and the instructions for administrators are
designed to maximise validity by focussing only on the macroskill being
assessed. For instance, the receptive skill of listening requires some sort of
productive response, but the framing of the task needs to make sure that
students’ productive skills are not being assessed. Certainly, “it is not easy
Sample placement test_Jackie Cipollone_110316.doc Page 7 of 29
to design listening comprehension tests which mirror the purposes of real-life
listening” (Brindley, 1998,:174). So, in the interview component the
administrator provides contextual support by referring to the attendance
pattern options and having the student listen and do minimal writing as a
response to listening. It is also recognised that, as van Lier points out
(1989), an oral proficiency interview is not a conversation; the interview is
not intended to be a conversation, but more a face-to-face interpersonal
exchange. The interview component of this placement test is a transaction
between the teacher and the prospective student. It has only one of the
features which van Lier says are “the basic characteristics of conversation”
(p. 495): it is face to face. It is highly planned and will be largely
predictable, and there is no “equal distribution of rights and duties” (p. 495).
The trade-off for the absence of these characteristics is that it is authentic
language use for its purpose, and therefore has construct validity.
Limitations are recognised The chief weakness of this test is its reliability,
and this is directly related to its validity. Given the low stakes nature of the
test, this is considered an acceptable trade-off. However, interviewer
training will be conducted with the aim of increasing the test’s reliability.
Nonetheless, inter-rater reliability is a real issue of practicality when there is
a large staff turnover as a result of an increasingly casualised workforce.
Another aspect of the test related to reliability is the fact that a pre-recorded
listening text is not used and there will inevitably be individual differences
between interviewers as well as within the same interviewer on different
Sample placement test_Jackie Cipollone_110316.doc Page 8 of 29
occasions. In an attempt to increase the test’s validity and authenticity, a
pre-recorded input text was not considered appropriate.
Cumming notes, in relation to assessing students’ writing, “Because of the
reliance on human interpretation in rating compositions impressionistically,
this is perhaps the major area of controversy in writing test development and
administration at the moment” (1997:55). This impressionistic characteristic
is evident in assessing any language macroskill in a communicative way. It
is particularly an issue in assessing the productive skills of speaking and
writing. This is why a marking guide has been developed, to be consulted
after the student has left the interview, as a means to increase the test’s
reliability by referencing student performance to ISLPR descriptors.
When the first draft of the test was completed, it was checked by one Head
Teacher and three teachers, all of whom are native English speakers. The
only item that was of concern was question 6 in the written component,
which asks the student to describe the day’s weather. It was noted that this
mars the communicative flavour of the test, and the suggestion was made to
omit it. The question was chosen on the grounds that no prior content
knowledge is required to answer it, as all students will have witnessed the
day’s weather on their way to the testing venue. It is likely to be a question
that has not been routinely asked of students before, so should provide some
novel sentences in response. However, teachers who pilot the test will be
Sample placement test_Jackie Cipollone_110316.doc Page 9 of 29
asked to comment on its usefulness, and if it is found to be of no value it can
be omitted from further tests.
Description of the administration
Before the testing day, teachers need to be familiarised with the format of
the test and the marking guide. On test day, prospective students will be put
together in a room or rooms on arrival, where a teacher will hand out the
written component (Appendix 2) and explain the purpose of the information
gathering. Students will be given 20 minutes to complete the written
component. The teacher in the room can see which people finish first, and
they can then be given a numbered ticket as they finish, or have their names
noted. The numbering system or order of names can be used to call them in
for the interview component.
At the interview, the teacher will greet the student and have a look at the
written responses. This will give experienced teachers a good idea of the
general written proficiency of the interviewee, and the content can be used to
draw out a spoken response from the student. If, from the written response
and initial oral interaction, it seems that the student is either clearly below or
clearly above ISLPR 1, the teacher can use supplementary higher or lower
material (not provided here) if s/he feels that this will be required to help
assess the appropriate class level. The teacher will then explain the various
attendance pattern options, and ask the student to indicate their preference
on the form (provided here in Appendix 3), which will then be stapled to the
Sample placement test_Jackie Cipollone_110316.doc Page 10 of 29
written sheet. Following that, the teacher will explain that the student needs
to return on another day to see if s/he is on a class list or a waiting list. The
teacher will then show the student the map and details (in Appendix 3), and
ask them to write the details on an appointment card. If the student’s
proficiency means that s/he is unable to do this, the information will be
provided to them in written simple English. The teacher will ask the student
if s/he has any questions, and the interview will close.
After the student has left, the teacher will confirm the initial impressions of
the student’s proficiency against the marking guide (Appendix 4), and then
assign a class.
The test should be piloted at three locations. An attempt to validate the test
should occur in the first three weeks of tuition, which will be “an enquiry …
into the proportion of students who were thought to be misplaced” (Hughes,
2003:30). No statistical analysis need be undertaken but, rather, qualitative
feedback can be sought on the test’s predictive validity. Feedback can be
gathered on the test’s administration and content, and any required
amendments will be made. For instance, if teachers feel that higher or lower
supplementary materials need to be developed, this can be undertaken.
The test can then be offered to other sites on request. Selected teachers will
need to be trained in the administration of the test, and they can then train
Sample placement test_Jackie Cipollone_110316.doc Page 11 of 29
their colleagues. A validation according to number of misplacements will
again need to be determined.
Sample placement test_Jackie Cipollone_110316.doc Page 12 of 29
References
Alderson, J.C., C. Clapham and D. Wall 1995. Language test construction and
evaluation. Cambridge: Cambridge University Press. Brindley, G. 1989. Assessing achievement in the learner-centred curriculum.
Sydney: National Centre for English Language Teaching and Research. Hughes, A. 2003. Testing for language teachers. Cambridge: Cambridge
University Press. Spence-Brown, R. 2001. The eye of the beholder: authenticity in an
embedded assessment task. Language testing 18, 4: 463-481. van Lier, L. 1989. ‘Reeling, writhing, drawling, stretching, and fainting in
coils: oral proficiency interviews as conversation’ in TESOL Quarterly, 23, 3.
Weir, C.J. 1993. Understanding and developing language tests. London:
Prentice Hall. ________ 1997. ‘The testing of reading in a second language’. In Clapham,
C.M. and D. Corson (eds.), Encyclopedia of language and education. Volume 7: Language testing and assessment, 39-49.
Wylie, E. and D. Ingram 2007. International Second Language Proficiency
Ratings: General proficiency version for English. Queensland: ISLPR Language Services.
Sample placement test_Jackie Cipollone_110316.doc Page 13 of 29
Appendix 1
Placement test – information for teachers
Sample placement test_Jackie Cipollone_110316.doc Page 14 of 29
Introduction
This placement test is designed to assist teachers in placing new students in
either the Certificate I in ESOL (Access) or the Certificate II in ESOL. The
tasks are pitched at around ISLPR 1, the latter being the entry requirement
to the Certificate II. The idea is that the tasks will allow students to
demonstrate the upper limits of what they can do. Some students will only
be able to complete up to a certain point, or will demonstrate productive
skills below ISLPR 1. Others may complete the written component very
quickly and will demonstrate productive skills above ISLPR 1.
Supplementary material could be provided if it is apparent in the
administration of the test that a prospective student is either above or below
approximately ISLPR 1. The supplementary material will help determine if
the student needs to be in one of the Certificate I streams, or if they should
consider one of the courses above Certificate II.
The test comprises a written component, followed by an interview.
The written component focuses chiefly on establishing the intending student’s
writing skills, though it also provides some indication of the student’s reading
skills as well. It will also help identify students’ future goals.
The interview component is used to help (a) indicate the student’s listening
and speaking skills, (b) confirm their reading skills and (c) establish their
goals.
Like the courses in which the students will be enrolled, the placement test
uses and seeks to elicit language for real life purposes. It is a direct test of
students’ abilities (it requires them to speak, listen, read and write) and
reflects the communicative language methodology they will encounter in their
classes. The test seeks to establish what students can do, rather than what
they cannot do.
Sample placement test_Jackie Cipollone_110316.doc Page 15 of 29
The test is designed to be administered and completed within one hour: 20
minutes for the writing; 20 minutes for the interview; 20 minutes for the
teacher to finalise assessment and determine an appropriate class.
The test will be considered effective if there is minimal inappropriate
placement in classes.
Content of main written component
Reading and writing
Ability to respond to requests for basic personal information by
o completing a form
o responding to a self-assessment checklist related to own learning
needs and/or goals
o writing very short series of original sentences
Content of interview
Speaking and listening
Ability to sustain a simple face-to-face interaction by
o expressing personal preferences
o giving simple follow-up information
o listening and responding to details about an appointment
Reading and writing
Ability to read a notice and identify location, date and time of the event
Ability to write details of event for own future reference
Sample placement test_Jackie Cipollone_110316.doc Page 16 of 29
Students who complete the written component and respond to the questions
in the face-to-face component as per the marking guide can be placed in the
Certificate II course.
If it is apparent from the written component that students are below ISLPR 1,
it may be necessary to give them easier supplementary material to help
determine which of the Certificate I streams is most appropriate for them.
If it is apparent from the written component that students are above ISLPR
1, it may be necessary to give them more challenging supplementary
material to help determine which of the Certificate III courses is most
appropriate for them.
However, the responses to this placement test may be such that interviewers
are able to determine the most appropriate class using it alone. This will
relate to the prospective student’s performance, and the decision to use it
alone will be at the teacher’s discretion.
Sample placement test_Jackie Cipollone_110316.doc Page 17 of 29
Administration of test
Written items – approximately 20 minutes
Give prospective students approximately 20 minutes to complete the
written component. Students who finish sooner can be called to interview
first. Note that they may finish quickly because they are not able to write
much, or they are more proficient than ISLPR 1.
Interview component – approximately 20 minutes
Greet the student appropriately. Make a mental note of their basic
courtesy forms and their register awareness.
Tease out information provided in writing by asking questions such as
“How did you find out about TAFE?”, “Do you like living in Anytown?”,
“How did you travel here today?”, “Do you play any sports?”, “Tell me
about your favourite television show”. Some students will have had
distressing experiences prior to their arrival in Australia, so steer clear of
questions about family members or work and education in home country
(unless the student brings them up independently).
Tell student about possible enrolment patterns, and ask for preferences.
Note ability to respond to this.
Then give enrolment and requirement checklist and ask student to
complete by ticking boxes. Do this by using a two or three-step
instruction. For instance, have two different coloured pens on the table
and say “Use the blue pen to complete this form, then give it to me. Just
tick one box for each question.” Note the response.
Show the student the map and details (suggest this page be laminated),
and ask them to write the details on an appointment card. If the
Sample placement test_Jackie Cipollone_110316.doc Page 18 of 29
student’s proficiency means that s/he is unable to do this, give them the
information in simple written English.
Ask the student if s/he has any questions, and close the interview.
Assessment – approximately 20 minutes
After the prospective student has left the room
Record the class you think is most appropriate for the student, based on
his/her current proficiency and learning goals, in the box on the first page
of the marking guide.
, use the marking guide to
inform your assessment of his/her current proficiency. Students at ISLPR
1 should get ticks for most boxes. Use other information provided, such
as learning goals, to determine focus of class for placement.
Staple written component and completed enrolment and requirement
checklist to back of completed marking guide. This will be filed for the
eventual class teacher’s information.
Sample placement test_Jackie Cipollone_110316.doc Page 19 of 29
Appendix 2
Placement Test – written component for prospective students
Sample placement test_Jackie Cipollone_110316.doc Page 20 of 29
ANYTOWN COLLEGE OF TAFE
LANGUAGES SECTION
Thank you for coming today.
We will ask you to answer some questions about yourself. Your answers will
help the teachers choose the best class for you.
Please complete all three pages and then wait for a teacher to call your
name.
Name
Address
Phone number
Home language
Years in Australia
Please read these sentences and tick () one
1. I learned English before:
box for each sentence.
in my country.
at AMES.
at work.
at home.
Sample placement test_Jackie Cipollone_110316.doc Page 21 of 29
2. I want to learn English because:
I want to get a job.
I want to talk to Australian people.
I want to study at TAFE or university.
3. About my English:
My English reading and writing are better than my speaking and
listening.
My English speaking and listening are better than my reading and
writing.
Please write sentences to answer these questions.
4. What do you read in English?
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
5. What do you listen to in English?
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
Sample placement test_Jackie Cipollone_110316.doc Page 22 of 29
6. Describe today’s weather.
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
7. Please write anything more about yourself that you want the teachers
here to know.
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
…………………………………………………………………………………………………………………..
Thank you.
Now please wait for a teacher to call your name.
Sample placement test_Jackie Cipollone_110316.doc Page 23 of 29
Appendix 3
Materials for interview component
Sample placement test_Jackie Cipollone_110316.doc Page 24 of 29
Part-time (10 hours per week)
Three mornings
Two mornings and one afternoon
Two evenings
Full-time (20 hours per week)
Five days
Four days
I require child care while I study here
Yes
No
I require a parking spot while I study here
Yes
No
Other comments: …………………………………………………………………………………………….
……………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………..
Sample placement test_Jackie Cipollone_110316.doc Page 25 of 29
Class lists will be put up in Building Y on Monday 2 July at 12:30 pm.
Come back on that day to see if you have a class.
If your name is on a waiting list, you will need to come back on Wednesday 4 July at 9:30 am.
LUMLEY STREET OFFICES, “Y” Block, 13 – 15 LUMLEY STREET, GRANVILLE NSW 2142
Sample placement test_Jackie Cipollone_110316.doc Page 26 of 29
Appendix 4 Marking Guide for teachers
Sample placement test_Jackie Cipollone_110316.doc Page 27 of 29
Student’s name Class
Tick skills demonstrated by prospective student.
Name
Written responses appropriate and indicate understanding of requests for information
Roman alphabet used, though use of case may be inconsistent
Basic punctuation conventions generally observed
Address
Phone number
Home language
Years in Australia
Please read these sentences and tick () one
Student followed this simple two-step instruction
box for each sentence.
1. I learned English before:
in my country.
at AMES.
at work.
at home.
Confirmed by student at interview
2. I want to learn English because:
I want to get a job.
I want to talk to Australian people.
I want to study at TAFE or university.
Confirmed by student at interview
3. About my English:
My English reading and writing are better than my speaking and listening.
My English speaking and listening are better than my reading and writing.
Confirmed by student at interview
Sample placement test_Jackie Cipollone_110316.doc Page 28 of 29
Significant features of listening and speaking at ISLPR 1
Responds appropriately to basic questions and statements
Responds appropriately to two-step instructions and/or short compound sentences
Can sustain a number of ‘turns’ in spoken discourse
Limited vocabulary range
Hesitant production, with some pausing
Extremely tentative register flexibility
Please write answers to these questions.
4. What do you read in English?
5. What do you listen to in English?
6. Describe today’s weather.
7. Please write anything more about yourself that you want the teachers here to know.
Significant features of writing at ISLPR 1
Sentences are complete
Sentences comprise subject, verb and object/complement/very short phrase
Uses only basic, high-frequency connectives
Grammatical errors may cause misunderstanding
Written form is recognisable
Sample placement test_Jackie Cipollone_110316.doc Page 29 of 29
Part-time (10 hours per week)
Three mornings
Two mornings and one afternoon
Two evenings
Full-time (20 hours per week)
Five days
Four days
I require child care
Yes
No
I require parking
Yes
No
Oral instructions followed
Preferences and requirements indicated
Other comments:
Sentences are complete
Sentences comprise subject, verb and object/complement/very short phrase
Uses only basic, high-frequency connectives
Grammatical errors may cause misunderstanding
Written form is recognisable
Map exercise – reading and listening
Identifies location, date and time of event
Recommended