Upload
primalee-encarnacion
View
190
Download
0
Tags:
Embed Size (px)
Citation preview
2. This last realization led directly to the development of
criterion referenced tests that were based on the objectives of our
courses. These tests were developed by the teachers in each skill
area to directly reflect the objectives of their courses and the
types of activities that were going on in those courses. Therefore,
the test could be considered very much like the final examinations
that teachers almost everywhere develop for their courses.
3. We set out to create two forms of the test for each course we
could use them as a pre-test and post-test in each session. We
aimed at having three forms of each rest for each course so that we
could use them as pre-test, midterm, and post test. When these
tests were fully implemented, we were in a position to give the
students diagnostic information objective-by-objective based on pre
test, diagnostic progress objective-by-objective based at the
midterm, and achievement information objective-by-objective at the
end of the course.
4. One problem we had to deal with was the practice effect. Brown
(1988, p.35) defines the practice effect as the potential influence
of the measures on each other. In this case, simply having taken
one of the tests was likely to improve the students performances on
any subsequent administrations of the same test. We took the view
that the tests should be considered part of the teaching and that
the practice effect could be considered part of the learning
experience. We wanted to minimize the effect that remembering
specific test questions would have on subsequent tests. We did not
want any student taking exactly the same form of any test more than
once.
5. To minimize the impact of the practice effect, we used what is
called a counterbalanced design for administering the tests.
We developed such an elaborate system of tests because we felt that
it was important to provide valuable diagnostic, progress, and
achievement information to the students and their teachers, and to
do so without causing a direct practice effect by allowing students
to take the same test twice. We also recognized that it is
important to have pre test-post test information in developing CRTs
so that the effectiveness of the items can be studied and the tests
can be improved.
6. Benefits of a Sound Testing Program
7. These criterion-referenced tests were not easy to develop. In
fact, a tremendous amount of work was involved. However, a
determined group of teachers did create these tests, and did so in
multiple forms. The payoff from all this work was enormous,
especially with regard to gathering information on all the elements
of the curriculum design process.
8. First, the tests helped us to closely examine our perceptions of
the students needs.
Second, after we discovered that some objectives did not need to be
taught, we had the freedom to concentrate instead on the remaining
objectives or to add new objectives designed to meet more advanced
needs. Perhaps we were succumbing to what Tumposky (1984)
sarcastically calls the cult of efficiency, but frankly I do not
understand why attempting to be relatively efficient is a bad
thing.
9. Third, changing the objectives due to what we learned from the
tests naturally led to rethinking our materials and teaching
strategies to meet the newly perceived needs of the students.
The fourth and last benefit gained from our testing program was
that whenever we needed to focus on program evaluation, we had a
great deal of information ready to be presented. We had information
about the overall proficiency of our students, as well as
information about what and how much the students were learning in
the classroom.
10. Initial Screening Procedures
11. Before students are admitted to UHM, they are carefully
screened by the Office of Admissions and Records. The students
previous academic records, letters of recommendation, and TOEFL
scores are reviewed and only those students with total scores of
500 or higher are accepted for admissions to UHM. This information,
including each students TOEFL subtest and total scores, is
transmitted to the ELI. If their scores are above 600, students are
notified that they are exempted from any ELI requirement. Those
students who have scored between 500 and 599 are notified that they
must clear the ELI immediately upon arrival at UHM. In this way,
the initial screening procedures narrow the range of English
proficiencies with which the ELI must deal.
12. Placement Procedures
13. In most cases, however, students who scored between 500 and 599
on the TOEFL are required to take the ELI Placement Test (ELIPT) as
soon as they arrive on campus. We require this test for three
reasons:
Because we want more detailed information than is provided by TOEFL
scores
Because we want specific information about how the students will
fit into our particular language programs
Because we are also interested in getting information that is more
recent than their TOEFL scores
14. Placement procedures are particularly important in programs,
like our ELI, that have different tracks and level. Recall that we
have four tracks, each of which is focused on one skill. Within
these tracks there are up to three levels. So the placement
decisions must be focused on the skills involved.
The ELIPT is a three-hour test battery consisting of six subtests:
Academic Listening Test (ALT), Dictation (DCT), Reading
Comprehension Test (RCT), Cloze Procedures (CLZ), Writing Sample
(WTS), and Academic Writing Test (AWT)
15. The Academic Listening Test and Dictation are used to place
students into our listening skill courses. The Reading
Comprehension Test and Cloze Procedure are used for the reading
skill courses. The Academic Writing Test and Writing Sample are
employed for placing students into proper writing skill
level.
In each skill area, one of the subtests is discrete-point, that is,
it tests the discrete bits and pieces of the language, and one is
integrative, that is, it requires the student to integrate and use
a variety of different language skills.
16. First-Week Assessment Procedures
17. During the first week of instruction, all ELI teachers
administer a criterion-referenced test designed specifically to
test the objectives of their courses. The teachers are also asked
to keep a close watch on their students to see if any have been
misplaced. When teachers find students who seem to be in the wrong
level, they consult with the ELI director. Interviews with the
students are conducted to find out what they want to do, and they
are advised about what we think should happen.
18. Achievement Procedures
19. At the end of each semester, evaluation report forms are filled
out by the teachers about the performance of every student.
Teachers are asked specifically what ELI course level the students
should take during the next semester. On the basis of classroom
performance and the students score on the criterion-referenced
final achievement test, the teachers can suggest that students skip
one level or be exempted from any further study in that particular
skill area. Again, interviews with the ELI director are arranged,
and the students are advised on the course of action that we think
would be most appropriate.
20. Criterion-Referenced Diagnostic and Achievement
Procedures
21. There are also tests within each of the courses in the ELI
courses that are designed to test the specific objectives of that
course. Unlike the criterion-referenced tests developed for the
GELC program, the tests in the ELI only exist in two forms.
22. The two forms of each test are used in a counterbalanced design
for the course pre test and post test. These tests, like those at
GELC, have proven useful for giving information to the students and
their teachers, as well as for assessing the accuracy of our
perceptions of students needs and the appropriateness of the course
goals and objectives.
23. This whole system of tests, including proficiency, placement,
diagnosis, and achievement, is made relatively efficient by
instituting the role of lead teacher for a teacher who is given 50
percent release time to develop, analyze, revise, score, and report
the results of ELI tests.
24. For the criterion-referenced diagnostic and achievement tests,
the lead teacher has proven essential in rallying the teachers to
review and revise each of the forms for each course and getting the
tests to the teachers on time for use in class.
25. ELI Testing Program
26. We sincerely hope that the vast majority of the students who
are served by the procedures discussed above are correctly
classified, placed, diagnosed, and promoted. Decisions are made by
human beings, and even when they are based on seemingly scientific
information in the form of test scores, human judgments can be
wrong.
27. All this takes a little more effort on the part of the
administrators and teachers, but the benefits gained from effective
and humane testing procedures accrue to all students teachers, and
administrators alike.
28. One particularly useful side effect of having a complete
testing system, or program, is that information gained from one
type of test can sometimes be utilized to improve the other types.
Information gained from the CRT Achievement Tests can prove useful
in revising the placement procedures so that they more closely
match that which is being taught in the classrooms. Brown (1989b)
describes one such process. The CRT subtest of the ELIPT was made
to more closely match the kinds of approach, syllabus, techniques,
and exercises that actually go on in the reading courses. The
procedures and statistics involved in the process are explained
step-by-step in the article cited.
29. Yes!! The End?!
(ha-ha..)
THANK YOU^^
GOD Bless!
J.K.A.S