DOCUMENT RESUME

ED 353 820 FL 020 903

AUTHOR Stansfield, Charles W.; And OthersTITLE The Development of the Polish Proficiency Test. Final

Report.INSTITUTION Center for Applied Linguistics, Washington, D.C.SPONS AGENCY Office of International Education (ED), Washington,

DC.PUB DATE 29 Dec 92CONTRACT P017A10051NOTE 139p.PUB TYPE Reports Descriptive (141)

EDRS PRICE MF01/PC06 Plus Postage.DESCRIPTORS *Language Proficiency; *Language Tests; Listening

Comprehension; *Polish; Psychometrics;Questionnaires; Reading Comprehension; *Scoring;Standardized Tests; *Test Construction; TestInterpretation; *Test Items; Test Reliability; TestUse; Test Validity; Uncommonly Taught Languages

IDENTIFIERS *Polish Proficiency Test

ABSTRACTThe development of the Polish Proficiency Test, a

standardized, nationally-normed test of listening and readilgcomprehension for English-speaking learners of Polish, is reported.An introductory chapter provides background information about thetest's development, including discussion of the relationship betweenthe test and Polish language teaching, test structure and content,and test administration. The second section describes the start-up ofthe test development project and initial committee meetings. Section3 chronicles development of the field test form, including initialtest item selection and review and preparation of field testmaterials, and section 4 explains the field testing itself. Creationof test interpretation tables and a score scale is the topic of thefifth section, psychometric properties of the test, including testreliability, measurement precision, validity, and subscoreintercorrelations are detailed in section 6. A section on testoperationalization concludes the report. Appended materials includethe test booklet (introduction only), answer sheet, supervisor'smanual, examinee handbook, test interpretation manual, operationaltesting program forms, minutes of the first local developmentmeeting, item submission forms, examinee background questionnaire,standard error of estimation table, comprehension tasks for testitems, list of grammatical areas covered, sources for test passages,and test announcement. (MSE)

********************************************w**************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

***********************************************************************

The Development of the Polish Proficiency Test

Final Report for

Grant No. P017A10051

Office of International Research and Studies

U.S. Department of Education

Charles W. StansfieldProject Director

Dorry Mann KenyonAssociate Project Director

Xixiang JiangProject Coordinator

Center for Applied Linguistics1118 22nd Street, NWWashington, DC 20037

U S. MENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATION0 CENTER (ERIC)

Vdocument has been reproduced as

rece.ved trorn me person or orgarezatn.ongmaeng It

O Minor changes have been made to Improvereproduction Qualm?

40 Points of new or Opinions stated a, this doCu

CAmeal do not necessanly represent offic)atmm""..,.,,c,

9

December 29, 1992

2MT COPY ANIAILE

TABLE OP CONTEMTB

1. Introduction 11.1 Polish Language Teaching and the PPT 11.2 Structure of the PPT 21.3 Content of the PPT 51.4 Test Administration Time 71.5 PPT Test Materials 8

2. Project Start-Up 112.1 Test Development Committees 112.2 Initial Committee Meetings 12

2.2.1 Initial Local Advisory Committee Meeting . 122.2.2 Item Writer Training Workshop 142.2.3 Initial External Advisory Committee Meeting 14

3. Development of the Field Test F'rm :73.1 Initial Development of the Test Items 173.2 Review of the Test Items by the External Advisory

Committee 183.3 Preparing the Field Test Materials 18

4. Field Testing 204.1 Administration of the Test 204.2 Results of the Field Testing 224.3 Revision of the Test 24

5. Building Test Interpretation Tables and the PPT ScoreScale 275.1 Rationale 275.2 Building the Polish Proficiency Test Scale 285.3 Building the Test Interpretation Tables 29

6. Psychometric Properties of the Test 336.1 Test Reliability 336.2 Measurement Precision 336.3 Validity of the Test 356.4 Intercorrelations Among Test Subscores 39

7. Operationalization of the PPT 417.1 Information Dissemination During the Project . . . 417.2 Information Dissemination Following Project

Completion 42

List of Appendices

Appendix A: PPT Test BookletAppendix B: PPT Answer SheetAppendix C: PPT Supervisor's ManualAppendix D: PPT Examinee HandbookAppendix E: PPT Test Interpretation ManualAppendix F: Forms for the Operational Testing ProgramAppendix G: Minutes of First Local Development Committee

MeetingAppendix H: PPT Item Submission FormsAppendix I: Examinee Background QuestionnaireAppendix J: Table of Standard Error of EstimationAppendix K: Comprehension Tasks for the PPT ItemsAppendix L: Grammatical Areas Covered in the PPT Structure

ItemsAppendix M: Sources for PPT PassagesAppendix N: PPT Announcement

4

1. Introducti=

This report describes the development of the PolishProficiency Test (PPT), a standardized, nationally-normed test oflistening and reading comprehension for English-speaking learnersof Polish. The PPT was developed by the Center for AppliedLinguistics (CAL) under the auspices of the United States

Department of Education (Grant No. P017A10051) with the

cooperation of numerous linguists and Polish language expertsfrom a variety of academic institutions across the country. Theproject was initiated in October 1991 and completed in September1992. The test was developed in response to the lack of a

standardized proficiency test in the Polish language teachingfield and an increased need for such a test.

This introductory chapter provides the background to thedevelopment of the PPT, beginning with a description of therelationship between Polish language teaching and the PolishProficiency Test, and concluding with the description of thestructure, format and content of the Polish Proficiency Test.

1.1 Polish Language Teaching and the PPT

Before the project was proposed, a survey was conducted byCAL staff of tests available in Polish. The results indicatedthat the Polish language teaching field was handicapped by thelack of a nationally-available proficiency test. Only threePolish exams were identified in the CAL survey. One of theexams, the Lektorek Razor Blade Reading Proficiency Battery, is acomputer-based evaluation of reading and grammar. Owing to itsmode of administration, this Polish test is not accessible forinstitutions that do not possess comput.ized testingcapabilities. The second test was the Polish ProficiencyExamination, which was found to be more of an achievement testbased on the materials taught in the specific academic programwhich developed the test than a genuine proficiency test. Thethird was the Polish Second Language Graduation Proficiency Test.

1

5

The intended use of this test was as an exit criterion of the

foreign language requirement at the test developer's owninstitution. It was not intended for national use. Thus, noneof the three tests is easily accessible and available for usenation-wide.

The PPT now makes available to the Polish language teachingprofession a standardized and truly proficiency-oriented testthat is applicable to any Polish language program regardless ofits curriculum. The PPT can be used to meet a variety of needs

of Polish teaching programs, including the following:

Program Placement; i.e., institutions can use the test asa tool to determine the most appropriate instructional levels forincoming students with previous language training;

Program Evaluation; i.e., language programs can use thetest to assess their instructional effectiveness by examining therelative standing of their own students compared to nation-widetest performance;

Emit Criterion; i.e., individual programs can establishtheir own required exit score to help them determine if a studentis eligible to leave their program or to get credit for it;

Selection Criterion; i.e., schools, study abroadprograms, business or government agencies can use the test toselect the best applicants from a pool of candidates who have

acquired their knowledge of the language in a variety of ways;Credit by Ezam; i.e., institutions can offcr their

students the option of fulfilling a language requirement for acertain program on basis of the PPT score; and,

Research; i.e., the PPT can be used as a common measurein research projects.

1.2 Structure of the PPT

The PPT has three sections: Listening Comprehension,Reading Comprehension and Structure. All items on the PPT arefour-option multiple choice.

In the Listening Comprehension section, examinees hear a

2

passage in Polish, followed by a question about it in English.Both the stimuli and the question are heard from a

professionally-recorded Master Tape. After hearing the Englishquestion, examinees then choose one of the four answers printed

in English in the PPT Test Booklet and mark their choice on themachine-readable PPT Answer Sheet.

In the Reading Comprehension section, examinees arepresented with a variety of Polish texts. each followed by aquestion about it in English. The examinee is provided with fourresponse choices. The Polish text, the English question and the

four choices are all printed in the PPT Test Booklet.

The Structure section his two parts: filling in the blanks(cloze) and error identification. In Part One, there are five

Polish passages, each with five missing portions. The missingportions are indicated by an underlined item number. Beneatheach passage are five items labeled with item numberscorresponding to those in the passage. For each item, examinees

are required to choose one of the four options which completesthe corresponding missing portion of the passage in a meaningfuland grammatically correct way. In Part Two, examinees are

presented with three written Polish passages, each with fivenumbered lines. In each numbered line, there are four underlinedwords, marked A, B, C, D. Examinees are required to identify the

one underlined word which must be changed in order for the lineto be grammatically correct. Examples of these items areincluded in the PPT Examinee Handbook.

The design of PPT is intended to make it flexible. Aprogram may choose whether to administer the Structure sectiondepending on whether or not grammar is emphasized in instruction.

Although normally administered in a single session, if requiredby class scheduling constraints, the PPT may be administered intwo separate sessions.

Table 1 provides an overview of the organization of the

3

PPT.

Table 1Organization of the PPT

Total Total Number Format of NumberSection Time of Items stimulus of ItemsListening 40 min 50 Utterances 10

Dialogues 25Monologues 15

Reading 50 min 50 "Non-linear Text" 15Comprehension "Passages" 35

Structure 30 min 35 Cloze 25Error Identification 10

Although in general items on the PPT are arranged in order

of increasing difficulty, they are also grouped according to theformat of the item stimulus. For example, items in the Listening

Comprehension section are grouped into three parts according tothe format of the listening passage. Items in the first part,

"Utterances," present the examinee with the speech of a singlespeaker excerpted from a longer dialogue or conversation. Itemsin the second part present short dialogues between two speakers.Items in the third part present examinees with speech found innaturally-occurring monologues, such as those from radio

announcements or news broadcasts.

Although there are no separately designated parts in theReading Comprehension section, there are a variety of text typesthat are used. These may be classified into two main groups.The first is "Non-linear Text," which refers to reading stimulusthat is non-prose in format; i.e., whose form falls outside thenormal conventions of sentence, paragraph and text structure.

Examples include the writing found in signs, book titles, onstamps, in schedules, on identification cards and the like. Thesecond may be termed "Passages." These texts present examineeswith prose and expository text following normal conventions ofsentence and paragraph structure. The classification of thereading stimuli into these two categories is somewhat arbitrary,

4

6

since many passages contain both (e.g., newspaperadvertisements). In Table 1, an attempt has been made to

classify each stimulus text by its primary characteristic.Items in the Structure section are of a cloze (fill-in-the-

blank) type and error identification type. Both types focus onaspects of grammar and structure. The cloze type, which is PartOne, has five passages, each containing five test items. The

error identification type, which is Part Two, has two passages,each containing five test items.

1.3 Content of the PPT

The development of the PPT followed strict guidelines interms of the test content covered. All Polish materials on thePPT are drawn from authentic sources; i.e., Polish language

materials prepared for Polish native speakers. Sources forlistening passages included Polish movies, recordings ofconversations, and radio and television programs. For passagesfor reading and structure, sources included Polish magazines,journals, newspapers, books, informational brochures, merchandisepackaging, tickets, public signs and personal documents.

Every effort was made to include as wide a range as possibleof the various social and institutional interactions that wouldbe most likely encountered in real-life language-use situationsin a Polish-speaking environment. The content covered on the PPTcan best be described in terms of topic areas (subject of thelistening or reading passage). Table 2 presents an overview ofthe topic areas (content) of the PPT covered in the listening andreading sections of the test. The first column lists the topicareas covered, the second lists the number of listening passagesfor which that topic could be described as the main topic of thepassage, and the third lists the same for the reading passages.

5

Table 2

Topic Areas Covered on the PPT

Topic Area Listening Section Reading Section

1

Business

Consumerism

Dates 1

Economics/the Economy

Education

Entertainment 5 5

Everyday Life

Family

Food

Geography

Government

Health 5 3

History

Housing

Information Listing

Politics

Social Exchange 3

Social Life --

Society ,4

Sports

Transportation

Weather1

Other ---

6

1 U

Each passage in the listening comprehension section was also

classified by the test developers in terms of the primaryspeaking function of the passage. In many cases, there areseveral speaking functions. Each item, however, was assigned

only one primary function, the one most closely associated withgetting the passage's item correct. Table 3 gives an indication

of the distribution of the speaking functions found in the PPTlistening comprehension section.

Speaking

Table 3

Function in the listening Section

--,

Speaking Function Number of Items

Complaining

Describing 1

Expressing an opinion 2

Giving advice 1

Giving information 19

Making an inquiry 11

Interviewing 1

Making a comment 1

Making a request -1_

Making a purchase 1

Making an introduction 1

Making an invitation 1

Negotiating 1

iPersuading I

Showing preferences 1

1.4 Test Administration Time

The three sections of the PPT may be administered in either

7

11

one or two sessions, with the administration of the structure

section being optional. In a two-day session, the listening

comprehension section and the structure section (if included) are

administered at the first session, with the reading sectionadministered at the second. Exclusive of passing out test

booklets, filling in personal background information on the

machine-readable answer sheet, and taking care of other

administrative matters, the listening comprehension section

requires approximately 40 minutes, the reading comprehensionsection 50 minutes, and the structure section, 30 minutes. If

given in one sitting, the total testing time is about two hoursand a half. If administrative matters are taken care of before

the testing session begins, the PPT can be administered in one 70minute and one 50 minute class period. Two 80 minute periods areadequate for handling the complete administration of the PPT.

1.5 ?PT Test Materials

The operational program of the PPT consists of the followingmaterials (copies of these materials can be found in theappendices as indicated):

PPT Test Pooklet

The PPT Test Booklet contains all test instructions, theoptions for the Listening Comprehension items, the Polish text,English questions and options for the Reading Comprehensionitems, and the Polish text and options for the Structure items.Each examinee receives a copy of the test booklet, which iscollected by the test supervisor at the end of the test. Becausethe test is secure, only the cover of the test booklet is

reproduced in Appendix A: PPT Test Booklet Cover.

PPT Master Tape

The PPT Master Tape is a professionally recorded audio tapecontaining the listening stimuli and the English questions forthe Listening Comprehension section of '..he PPT. Normally, two

8

copies of this tape are sent to institutions ordering the PPT,

one to play and one as a back-up.

PPT Answer Sheet

This is a machine-readable answer sheet, specially designed

for the PPT program. In addition to spaces for examinee

responses to items, it contains areas to collect examinee

background data. A copy of the answer sheet can be found in

Appendix B: PPT Answer Sheet.

Test Administration Instructions (Supervisor's Manual)

This is a short manual for test supervisors containing all

the necessary instructions for administering the test. In order

to ensure that the PPT is administered under standardized

conditions, all instructions to examinees are to be read verbatim

from this manual. A copy of this manual is contained in Appendix

C: PPT Supervisor's Manual.

PPT Examinee Handbook

This handbook is intended to familiarize examinees with the

test before they take it. It reviews basic information about the

test and gives sample test items for the test. Each examinee

receives a copy of the Examinee Handbook from the test supervisor

prior to taking the test. A copy of the preliminary handbook is

contained in Appendix D: PPT Examinee Handbook. Before the

final copies are printed, CAL is waiting on some data from

several government agencies that will enable preliminary

conversion tables between the PPT and the ILR Proficiency Scale.

Otherwise, the Examinee Handbook is complete.

Test Interpretation Manual

A test interpretation manual has been prepared for the PPT.

The Test Interpretation Manual helps test users interpret test

scores so that they can be meaningfully used in their programs.

In addition to a brief description of the PPT, the Manual

9

contains technical information on the tests and test

interpretation tables. A copy of the preliminary manual is

contained in Appendix E: PPT Test Interpretation Manual. Again,

CAL is waiting on some data from several government agencies that

will enable preliminary conversion tables between the PPT and the

ILR Proficiency Scale before printing the final version.

Otherwise, the Test Interpretation Manual is complete.

Miscellaneous operational Test Forms

There are several forms that are used in the operational

testing program, sample copies of which may be found in Appendix

F: Forms for the Operational Testing Program.

Test Order FormThis form is used by test users to order the test fromCAL.

ChecklistThis form is used to ensure that PPT test materials arereturned to CAL.

Supervisor's Report FormThis form is used by test supervisors to record anyproblems that may have occurred during testadministration.

10

2. Project Start-Up

The initial phase of the PPT project involved a great manyefforts to ensure a sound theoretical and methodological

framework for the development of the test items. This chapter

describes the initial steps of the test development project. It

encompasses both logistic issues and theoretical concerns.

2.1 Test Development Committees

The day-to-day activities of the project were coordinatedand directed by a team of CAL staff members. Charles W.

Stansfield served as the Director of the Project, with Dorry MannKenyon serving as Associate Project Director and Xixiang Jiang asProject Coordinator.

To assist in the development project, two test developmentcommittees were formed. The first was the External Advisory

Committee, consisting of Polish linguists and Polish languageexperts from across the nation. Many of these were professors inmajor university-level Polish programs. Below is a list of themembers of this committee and their affiliations.

External Advisory Committee

Waldemar MartyniukBarbara NiemczykLeonard A. PolakiewiczRobert A. RothsteinAlexander SchenkerOscar E. SwanKrystyna WachowiczJoanna Radwanska Williams

Stanford UniversityDickinson College

University of MinnesotaUniversity of Massachusetts

Yale UniversityUniversity of Pittsburgh

Defence Language InstituteState University of New York

Members of this committee were asked to review

specifications for the test, review the test forms before fieldtesting, and to review revisions made to the test form. Aspossible, these committee members also helped coordinate fieldtesting of the PPT at their respective institutions.

The second committee was the Local Development Committee.Members of this committee were Polish language specialists and

11

1.5

experienced language instructors. Members of this committee and

their affiliations are listed below.

Local Development Committee

Dobrochna E. Dyrcz-Freeman*Rafal M. Kwiatkowski*Grazyna Z. PrivorotskyKrystyna M. SadlowskaWaldemar E. Walczynski

(Formerly) Princeton University(Formerly) CAL

CALForeign Service Institute

CAL

* primary test item writers

Members of the local development committee met to draft the

initial test specifications, developed the test items, reviewed

items under development, reviewed the test form before field

testing, and served as consultants when special problems arose.

The two item writers were trained by CAL staff. They werein regular contact with CAL staff throughout the project. They

were tasked to find suitable listening and reading passages from

authentic sources and draft items to test listening and reading

comprehension, as well as to draft the structure items.

In addition to these two working committees, four Polish

native speakers were involved in recording the PPT Master Tape.

Given their various areas of expertise and experience in

teaching, testing and with the Polish language, all members of

the project committees contributed to its successful completion.

2.2 Initial Committee Meeting2

Each committee met within two months of the project start-up. The results of their initial meetings, which set the courseof the project, are described below.

2.2.1 Initial Local Advisory Committee Meeting

The Local Development Committee met in mid October, 1991, to

develop the test specifications, to set down the guidelines withwhich the test item writers would work, and to discuss issuesrelating to the development of the test items. Several test

development concerns were addressed at the meeting regarding

12

target levels, test domains, test format and the number of items

in each section. Members of the Committee conferred for a day

and concluded with a general agreement on the issues considered.

In view of the difficulty of the language itself and the

limited nature of the existing teaching materials, it was

recognized that the target level of the test should be between 0+and 2+ on the ILR scale, corresponding to Novice-Mid/High toAdvanced-High on the ACTFL scale. In consideration of the factthat most Polish language teaching programs do not continue

beyond the second year and that students in area studies at the

graduate level usually enter beginning Polish programs at the

undergraduate level, it was agreed that the envisioned target

examinee group for this test include Polish language students intheir first or second year of college and government or business

professionals in intensive language training programs.

It was also agreed that the PPT, as a proficiency test,

would measure the examinees' ability to function in a real

Polish-speaking environment. Therefore, it would not be designed

to accommodate any specific Polish language teaching text orcurriculum, nor would any sort of achievement test be appended tothis proficiency test. Acknowledging that the knowledge of

syntactic structures is an integral part of communicative abilityin the Polish language, it was determined that this test wouldinclude a structure section. In addition, it was acknowledged

that the materials used in the items should be authentic; i.e.,the type of language materials that are produced by native

speakers for use by native speakers in their native environment.The authenticity of the test was expected to promote and lead thetrend in the language teaching field toward the use of real

language to achieve communicative competence in the targetlanguage.

In terms of the test item format, it was conceded at themeeting that a four-option multiple choice format be usedthroughout the test, in conformity with the current trend oflarge scale tests. It was also agreed that Structure section be

13

comprised of two parts: a cloze format and an error

identification format. As for the length of the test, it was

suggested that the whole test contain between 125 and 150 items,

with roughly equal numbers in listening, reading and structure

sections.

Apart from the major issues summarized above, the Committee

also deliberated on other matters, such as listing potential

sources of authentic materials and making a list of major Polish

language programs. A copy of the complete minutes of the

o-:eting may be found in Appendix G: Minutes of First Local

Development Committee Meeting.

2.2.2 Item Writer Training Workshop

CAL staff designed and conducted a one-and-a-half day

intensive training program for the two primary item writers. The

training program offered the item writers an opportunity to

familiarize themselves with the test specifications and general

guidelines stipulated by the advisory committees for the project.

Charles W. Stansfield, Project Director, instructed the group on

specific issues in relation to structure and reading item

development. Dorry Mann Kenyon, Associate Project Director,

supervised the actual practice writing of test items.

This training enabled the item writers to apply their

knowledge of Polish and experience in teaching to developing test

items according to the stipulations set forth for this specific

project. Since both the item writers were experienced Polish

teachers, their work and joint revision on items under

development ensured the good quality of the test items.

2.2.3 Initial External Advisory Committee Meeting

Members of the External Advisory Committee were sent copies

of the Minutes of the Local Advisory Committee Meeting for reviewand comment. In addition, members of the committee met for a 4-

hour meeting scheduled during the annual conference of the

American Association of Teachers of Slavic and East European

14

Languages (AATSEEL) in San Francisco, California on December 28,1991. The main task of the meeting, besides introducing the PPT

project to the members and discussing the design of the

structure items, was to review drafts of the listening andreading items. Prior to the meeting, every member was mailed a

copy of selected items and asked to note down on revision forms

their individual comments. Thus, every item was reviewed by two

committee members prior to the meeting. At the meeting, the itemreview process was carried out as follows. First, the two

individuals assigned with the review of a test item voiced theircomments and opinions on it. Then, any member of the group had

the opportunity to comment on that item. In this way the itemswere reviewed one by one. All revisions and relevant comments

were marked on a single master copy of the items.

From the item review process and discussions among theadvisory members, the following concerns and topics of discussionemerged:

1. On the whole, the group felt the test items were very welldesigned.

2. The group felt that the listening items, as a group, seemedto be easier than the reading items.

3. The group felt that more easier reading items needed to beadded, and that there should be fewer items taken fromnewspapers.

4. The group stressed the need to produce a high qualityrecording of the listening passages, using a variety ofvoices.

5. The members felt comfortable with the idea of trying outboth the filling-in-the-blanks and the error recognitionformats in the structure section, even though they saiderror recognition is not usually done in classrooms.

The members of the External Advisory Committee reached ageneral consensus regarding the practicality and suitability ofthe test specifications and overall guidelines as recorded in theminutes of the Local Development Committee Meeting. They also

15

expressed their willingness to offer any assistance needed forthe project.

16

3. Development of the Field Test Form

After the initial start-up of the project, attention wasfocused on the development of PPT test form for field testing.This chapter describes the steps in developing the field testform.

3.1 Initial Development of the Test ItemsThe two item writers started writing the reading items first

because of the immediate access to sources of materials.Listening and structure items were prepared later, when movies,tape recordings and other sources materials were located. FromOctober to December, 1991, item writers worked on their ownschedules and submitted draft items weekly.

To standardize the item development procedure and to ensurethe quality of the items, CAL staff designed an item submissionform to help guide the item writers in the writing process. Itemwriters were instructed to first find authentic listening orreading passages suitable for testing comprehension. They werethen to focus on an appropriate aspect of each passage to testcomprehension in listening or reading. They then developed thequestion, chose appropriate distractors, and submitted the itemsto CAL for review.

Each item was submitted on a submission form that requireditem writers to classify and categorize it. A copy of this form(one for listening and one for reading) may be found in AppendixH: PPT Item Submission Forms.

After items were revised among the item writers themselves,they were reviewed by CAL project staff and either accepted,discarded or returned to the original item writer for furtherrevision. Accepted items were put into the PPT item bank.Regular checks were made to ensure that the items being acceptedwere fulfilling the specifications for the whole test in terms oftheir targeted level of difficulty, content (i.e., topic area)and speech function.

17

21

3.2 Review of the Test Items by the External Advisory CommitteeWhen the item bank contained more items than required by the

test specifications, items were then sent to the members of theExternal Advisory Committee for further review. Each committeemember was assigned a subset of the items, all from one sectionof the test. All items were reviewed by two committee members.On the basis of input received from the Committee meeting, heldduring the AATSEEL annual conference in San Francisco, items wereeither kept as they were, revised as per specific comments, orremoved from the item bank.

The structure items, which were prepared after the meetingat AATSEEL, were sent to individual members of the ExternalAdvisory Committee for review. Their comments were returned toCAL and incorporated by the item writers into the items.

After the draft test form was revised on the basis of inputfrom members of the External Review Board, the field testmaterials were prepared.

3.3 preparing the Field Test MaterialsFrom the items remaining in the item bank, CAL staff put

together the Field Test Form, following the guidelines set forthin the test specifications. Recognizing that items might berejected after field testing, this form contained more items thanwere envisioned for the final form.

The PPT test booklet for the field testing was preparedusing WordPerfect 5.1. A special keyboard was designed forsections containing Polish text.

To ensure that the listening passages were all clear and ofprofessional quality, they were re-recorded using native speakersresident in the Washington, D.C. area. Two male and two femalevoices were selected. The speakers were instructed on how toensure the authentic delivery of the listening passages. Thespeakers strove to read the written script in as natural a way aspossible, while keeping in mind the need for clear articulation.During the recording session at a professional studio, the

18

22

Associate Project Director served as the director, supervisingthe recording procedure. Two other individuals, one an advanced-level student of Polish, the other a Polish language teacher,listened critically to each passage as it was recorded. Thesetwo persons either agreed that each take was of appropriatequality for inclusion on the test, or gave input into improvingthe retake. English sections of the Master Tape (testinstructions and questions on the listening passages) wererecorded by a professional announcer. The recordings were editedby professionals at the recording studio.

The final test booklet for field testing contained 65 itemsin the Listening Comprehension section, 56 items in the ReadingComprehension section, and 40 items in the Structure section. Inaddition to the test booklet and master tape, other materialswere prepared for the field testing. These included thefollowing:

A machine-readable standard answer sheetAn instruction manual for test administrationAn examinee background information formAn examinee familiarization sheet

When this process was completed, the PPT was ready to befield tested.

19

2

4. yield Testing

In order to ensure that the developed PPT form was valid andappropriate for the group 'Jr which it was designed, the PPT wasfield tested between February and April of 1992 on examinees fromPolish programs in the United States and Canada. This chapterdescribes the field testing procedures and the results.

4.1 Administration of the Test

The PPT was designed to be appropriate for students in therange from Novice-High to Advanced-High on the ACTFL scale.Since language programs do not classify themselves according tothe ACTFL scale but in terms of years and credits, project stafffelt that the test should be appropriate for students from theend of a first year college level course to students in a thirdyear course. It was desired to include examinees at both thelower and higher ends of this range in the field testing samplein order to compare examinee performance across the entire scale.

CAL developed a list of all institutions in North Americathat indicated they offered Polish instruction. This was donewith the help of members of both the local and externalcommittees, and through the use of two directories published bythe American Association for the Advancement of Slavic Studies:Directory of Programs in Soviet and East European Studies 1990-1992 and Directory of Members 1991-93. Every institution wascontacted and invited to participate in the field testing. Inthis process CAL encountered two practical problems. The firstwas that a number of schools listed in the directories haddropped programs due to budgetary constraints. The second wasthat many of the programs were staffed by part-time instructorswho were not in a position to manage or coordinate formaltesting.

In the end, instructors from 15 different institutionsparticipated in the field testing of the PPT. At the Universityof Manitoba (Canada), the instructor administered the PPT to

20

24

students at other institutions, including high school programs.A total of 129 examinees participated in the field testing.Table 4 indicates the number of examinees completing at least onesection of the test together with their institution or, in thecase of the University of Manitoba, that of the test coordinator.

Name of Institution

Table 4Field Test Participants

state Numbe- of Examinees

American U DC 4Columbia U NY 13Harvard U MA 12McMaster U CANADA 2Stanford U CA 6SUNY, Stony Brook NY 7U of Manitoba CANADA 22U of Chicago IL 8U of Pittsburgh PA 3U of Minnesota MN 9U of Massachusetts MA 4U of North Carolina NC 11U of Connecticut CT 7U of Florida FL 16Villa Marie College NY 5

Total129

Examinees completed a background questionnaire before takingthe field test version of the PPT (see Appendix I: ExamineeBackground Questionnaire). Percentages used in the followingdescription are based on the number of examinees answering eachquestion. Fifty percent of the participating examinees were maleand 50% were female. Ninety percent were at a college oruniversity, 5% were in a public high school, and 5% were studyingPolish at a community school or other institution. Sixty percentreported that they are ethnic Polish. The majority of allexaminees (60%) indicated that they spoke no Polish at home. Ofall the examinees, 13% indicated that they spoke Polish at home"most of the tim&," 10% indicated that they spoke Polish at home

21

"less than half the time," and 17% indicated that they spoke

Polish at home "very infrequently."

In terms of the level of study at the time of the test

administration, the participants were enrolled in classes ranging

from the third year of a high school course to more than third

year college level Polish language course. Of those indicating

their level (117 students), the majority (61%) indicated that

they were in either the first or second semester of a first year

college level Polish course. 56; of all the students had not yet

completed one year (college or high school) of Polish

instruction, while 12% had completed four or morc years. Table 5

shows the level of study indicated by the field testing

examinees.

Table 5Levels of Polish Study of the Field Test Examinees

Level of Polish Instruction Frequency percentNone of the Options 6 53rd Yr High School 3 34th Yr High School 3 31st Semester or 1st Quarter 13 112nd Semester/2nd or 3rd Quarter 58 503rd Semester/4th Quarter 7 64th Semester/5th or 6th Quarter 12 105th Semester/7th Quarter 1 16th Semester/8th or 9th Quarter 5 4More than 3rd Year 2 8

Total 117' 101%b

'Missing = 12'Due to rounding

4.2 Results of the Field Testing

Both quantitative and qualitative data were collected on thefield test form. Quantitative data consisted of the examinees'

responses to the test items. Qualitative data consisted of

comments made by test supervisors, and, from one school,

22

26

examinees themselves.

Examinees recorded their responses to the backgroundquestionnaire and their answers to the test on the General

Purpose Answer Sheet form no. 4521 of the National ComputerSystems (NCS). Each sheet was scanned on CAL's NCS Sentry 3000Optical Scanner twice: the first time to collect background

information using the program Scantools, the second to score thetest using software MicroTEST Score II Plus, both by NCS. Thetwo databases thus entered were merged into one file using theParadox database system. Item analysis was conducted using theTest Analysis Program (TAP), a classical item and test analysis

program, and BIGSTEPS, a Rasch model IRT-based item and testanalysis program. Statistical analysis was performed using SAS.

Item analysis was first conducted on the entire group ofexaminees. Table 6 presents the summary of descriptive

statistics by section.

Table 6

Descriptive Statistics from the Field Testing

Number of Number Mean Std. Mean Std.Dev.Section geaminees of Items lam Reliability P-value P-valueListening 105 65 38.76

_psy,12.80 .93 .60 .19

Reading 118 56 36.32 11.65 .93 .65 .14Structure 127 40 22.36 9.13 .91 .56 .14

Table 6 shows that the subtest reliabilities were quitehigh, most likely a function of the length of each part and thefact that a wide range of abilities were represented in thesample. The mean p-value for the listening and reading items wasappropriate for a multiple choice test. The reading section wasthe easiest subtest, while the structure section was the mostdifficult.

The next step was to analyze the individual items to detectif any were malfunctioning. Information on the items wasacquired through TAP and BIGSTEPS. TAP analyses provided point-biserial correlations, p-values, frequency of responses to eachitem broken up by quintiles, graphic representations of the

23

percent of correct responses to test items by quintiles, and thenumber of examinees choosing each option, by quintile. BIGSTEPSprovided information on the fit of each item (and person) to anunderlying Rasch measurement model. Commonly accepted criteriawere used to identify problematic items: p-values that were below30; point-biserial correlations below .30; and Rasch fitstatistics above 2.0 or below -2.0.

This analysis revealed that the majority of the items on thePPT were functioning very well. The numbers of items that wereflagged by the above criteria were: 8 of the 65 listening items,8 of the 56 reading items, and 3 of the 40 structure items.

In addition to the statistical analysis, a qualitativeanalysis at the item level was undertaken based on writtencomments received from the test supervisors and from one group ofstudents. Moreover, the entire field test form was carefullyreviw ed by one member of the test development committee, whosecomae :s were also carefully considered. The vast majority ofcomments were highly favorable. On the item level, most of thecomments related to items that were flagged through thestatistical analyses. The information provided by both types ofdata corroborated each other. A few comments had to do withpersonal preferences of the test supervisors.

4.3 revision of the Test

After both the quantitative and the qualitative informationwas summarized, the local test development committee met todiscuss the field testing results and what to do about theflagged items. Since extra items were included in the field testform, the committee felt free to delete from the final form anyitems that were technically flawed rather than work on revisingthem. At the meeting, the committee agreed that the final testlength should be 50 items each for the listening and the readingsections.

In the listening section, 10 items were deleted by thecommittee on the basis of the quantitative and qualitative

24

C7T

analyses. Only one flagged item received minor revision that may

affect its quality: one of its distractors was revised so as not

to be as plausible as the original distractor was. One other

superficial change was made to two options in an item in order tomake all the options parallel. Other than these, no revisions

were made to any of the listening items.

In the reading section, three of the eight flagged items

were easier items with point-biserial correlations slightly below

.30. These items showed no misfit to the Rasch model. The

committee decided that these items should not be automatically

deleted. At this point they were all retained for their content.

Four clearly malfunctioning items were deleted, however. Minor

typographical errors were made to five of the items; three

affecting the Polish text, two affecting the English. The

committee felt that these minor changes would not affect thequality of the items.

In the structure section, the analysis indicated that three

of the five items in the last of the eight passages appeared tobe much too difficult for the field test examinees. Two of theseitems also showed misfit problems. It was decided to delete this

passage as a whole rather than to repair its items. Only two

other items showed minor problems, which were easily corrected.

Thus, the final version of the structure section contains 7

passages with 5 items each, for a total of 35 items.

Following the meeting, there were still 5 items from the

listening section and 2 from the reading section to be deleted inorder to bring the number of items in each section on the finalform down to 50. The selection of these items was made through

the use of the BIGSTEPS program. For each section, after the

initial run, misfitting persons and the items to be deleted wereomitted from a second run. The fit of the items was then

examined a second time, and the remaining number of items to be

deleted were selected on the basis of this analysis of fit.

Once the final items were selected, the recording studio

removed the deleted items from the tape for the listening

25

2S

comprehension section and a revised test booklet was prepared.

I

5. puildina Test Interpretation Tables and the PPT Score Scale

As reported in Chapter 4, CAL invited every known programlisted as offering Polish to participate in the field testing ofthe PPT. For various reasons, many programs could notparticipate in organized testing. All programs able toparticipate did so. This meant that no large pool of Polishexaminees who did not participate in the field testing wereavailable on whom to test the final version of the PPT at the endof the spring semester of 1992. Thus, in the fall of 1992, thefinal form of the PPT was being administered to over 30 examineesfrom invited government agencies in order to check for minorinfelicities. After their answer sheets are returned, the finalversion of the PPT Test Booklet will be printed.

Nevertheless, it was possible to estimate examinee abilityscores, compute test statistics, and to construct tables for testscore interpretation from the field test data. This process isexplained in this chapter.

5.1 Rationale

Using IRT (Item Response Theory), an examinee's ability isestimated independently of the items administered. This iscalled "item-free person measurement." An examinee's score isnot solely based on the number of correct answers on the test,but is empirically estimated based on the difficulty level of theitems the examinee answers correctly. Thus, in the context ofthe development of the PPT, an examinee's ability can beestimated based on responses to the entire set of itemsadministered in the field test. It can also be derived from anysubset of those items. In general, estimating ability from feweritems means a loss in the statistical precision of the abilityestimate. However, within statistical error, the abilityestimates based on the entire set or any subset will be the same.

Given this context, the ability of each examineeparticipating in the field test was estimated using the data from

27

the items remaining on the final form. This data was analyzedwith the BIGSTEPS computer program, which uses the Rasch IRTmodel. This data set, and the ability estimates from the Raschanalysis, were then used in constructing the preliminary testinterpretation tables for the PPT.

5.2 building the Polish Proficiency Test ScaleAs an aid in interpretation, the PPT Scale for reporting

scores was developed. After the Rasch analysis, all PPTexaminees had an ability score on the same scale. This abilityscore is no longer a "number right" score, but an estimate of theperson's ability along the continuum of the latent trait (theconstruct being measured) reported in terms of logits (a ratio innatural log odd units) centered at 0 (the average itemdifficulty) and extending (for the PPT) from about -2.5 to +5.An examinee's ability score in logits is defined as the point onthe item difficulty scale where the examinee has a fifty percentchance of getting the answer correct. Thus, an examinee with anability of 1.00 logits has a fifty percent chance of getting anitem with a difficulty level of 1.00 logits correct. Thisexaminee's chances of getting an item at 0.00 logit is greaterthan 50%, while for an item at 2.00 logits it is less.

This logit scale, however, can be changed by any lineartransformation without losing its linear quality. One standardapproach is to give the transformation a meaning in terms ofnorms. For the PIT, a "norming sample" was selected thatconsisted of 70 examinees from the 129 who participated in thefield testing. This was the total number of examinees whocompleted all three section of the PPT and indicated that theywere studying Polish at the college or university level. Thissample of 70 examinees is most likely typical of North Americanstudents studying Polish at the college level in any given year.At the time of norming, 67% percent of the group were in theirfirst year of Polish study, 24% were in their second, and 9% werein their th-rd (or higher) year. Forty-eight percent were male

28

32

and 52% were female. Fifty-eight percent were of Polishethnicity. Sixty-four percent of the group indicated that theyspoke no Polish at home, while 3% spoke Polish at home "most ofthe time," 6% spoke Polish at home "less than half the time," and27% spoke Polish "very infrequently" at home.

The mean score of this group in logits was assigned theeasily-remembered scale point of 100. One standard deviation inthe distribution of the scores of this group was set to be equalto 20 scale points. The PPT Scale score of 100 could then beinterpreted as the average ability score for the norming sampleof PPT examinees. An examinee receiving a score of 140 would betwo standard deviations above this mean; an examinee receiving ascore of 80 would be one standard deviation below this mean.

Below are the three equations used to transform abilityestimates in logits to scores on the PPT Scale. Scaled scoresare always rounded to the nearest integer.

Listening ComprehensionScale Score = 89.90 + (14.27xLogit Score)

Reading ComprehensionScale Score = 88.71 + (15.61xLogit Score)

StructureScale Score = 96.14 + (15.31xLogit Score)

5.3 Duilding the Test Interpretation TablesBecause of the small number of Polish examinees who could be

involved in the norming of the PPT, it was impossible to giveaccurate norms at every score level. For this reason, the primarytest score interpretation table is a table of quartiledistributions, shown in Table 7. This table provides quartilescores on the PPT Scale for examinees in the first, second, orthird or more year of Polish language study. The figures arebased on the results of all 129 examinees who participated in thefield testing of the PPT. However, not every examinee took everysection of the PPT. The characteristics of the entire group of

29

129 examinees did not deviate substantially from those of the 70examinees who comprised the norm group, described in Section 5.2.The 70-member norm group was a subset of the entire group of 129examinees.

Quartiles represent the bottom, second, third and topquarter of the score distribution. A score in the second quartileindicates that the examinee performed in the second quarter ofall students in his or her subgroup, and better than the bottom25 percent of all the examinees in that subgroup. A score in thefourth quartile indicates that the examinee scored in the top 25%of all other examinees in the subgroup; i.e., better than 75% ofall other examinees.

Table 7

Quartile Distributions for the PPT

by PPT Section and Year ofStudy

1

First Year Second Year Third Year or Higher

Quartile List Read Str List i Read Str List Read Str4th

(76-100%)

99 and

above

105 and

above

106 and

above

121 and

above

119 and

above

123 and

above

139 and

above

144 and

above

154 and

above3rd

(51-75%)

92-98 93-104 93- iu5 105-120 107 -118 110-122 130-138 127-143 122-153

2nd

(26-50%)

83-91 83-92 84-92 97-104 99-106 104-109 101-129 109-126 108-121

1

1st

(1-25%)

82 and

below

82 and

I below

83 and

below

96 and

below

98 and

below

103 and

below

100 and

below

108 and

below

107 and

below

I

I

Examinees (56) (60) (69) (18) (17) (19) 1 (6) (15) (15)

To use Table 7 in interpreting an examinee's score, the testuser first identifies the examinee as either a first, second, orthird (or higher) year student. Then, for each section, the userfinds the score group in the table that contains the examinee's

30

score and roads the quartile group from the left hand column thatcorresponds to the examinee's score.

In a few cases, examinees in the norm group indicated thatPolish was used in the home more than half the time. Suchexamines may have an advantage on a test of general languageproficiency. However, since these examinees formed only a smallminority of the total, and since they do comprise a part of thetotal group of North American students studying Polish, they wereincluded in Table 7. Nonetheless, PPT users may want tounderstand the meaning of an examinee's test scores in terms ofwhether the examinee indicates that much Polish il spoken at homeor not. Table 8 presents the average PPT Scale score for eachsection by year of study for examinees who indicated that they donot generally speak Polish at home (labeled English) and thosewho indicated that Polish is spoken about half the time or more(labeled Polish). Underneath each average score is the number ofexaminees upon which the mean is based. The more examinees, themore reliable the mean score is. Means for subgroups with lessthan 3 members are not shown.

Table 8Mean Scores on the PPT Scaleby Level and Home Language

(Number of Examinees in Parentheses)

Listening Reading,

StructureYear of Study English Polish English Polish English Polish

First Year 92(53)

---

(-)93

(55)96(5)

94(64)

96(5)

Second Year 107(15)

---(-)

107(15)

---(-)

110(16)

--(-)

Third Year(or higher)

124(5)

---

(-)119

(7)130(8)

121

(7)129(8)

To use Table 8, the test users finds the group to which the

31

examinee belongs and compares his or her score to that group'smean score.

The PPT is intended for English-speaking learners of Polish.Since the questions and options for the Listening and Readingsections as well as the directions for the entire test are inEnglish, knowledge of English is required to take the test. Thus,it is impossible to use these tests to measure the Polishlanguage proficiency of native Polish speakers (or speakers of alanguage other than English) who do not have a good knowledge ofEnglish. Test results may not be accurate for any examinee whodoes not have a good command of English; i.e., a level of Englishlanguage proficiency generally equivalent to that of a first-yearcollege student enrolled in the regular program of an Americanuniversity.

In the field testing of the PPT, 17 of the 129 examineesindicated that they were of Polish ethnicity and spoke Polish athome "most of the time." The responses of these examinees werecarefully examined to see to what extent, if any, theirproficiency in the English language may have been affecting theirperformance on the test. One analysis involved checking to seeif these examinees were getting easier items incorrect; i.e.,easy items that they may have missed because of their restrictedability in English rather than their proficiency in Polish. Noneof the responses of the Polish speakers showed any incongruitieson any of the three sections of the test. On the basis ofexperience to date, it is reasonable to assume that the level ofEnglish used in the test will not pose a problem to the nativePolish speaking examinee who has enough proficiency in English tobe enrolled at an English-speaking university in North America.

The information contained in this chapter and the followingis also presented in a more "user-friendly" manner in the PPTTest Interpretation Manual.

32

3t-,;

6. psychometric Properties of the Test

This chapter discusses the technical aspects of the PPT. Itmust be remembered that all the statistical analysis are based onthe results from the field testing. However, only performancedata on t!-e items retained in the final form of the PPT wereused.

6.1 Test Reliability

The reliability of a test is the extent to which it yieldsconsistent results. Thus, high test reliability is desirable.Table 9 gives the Rasch equivalent of the KR-20 reliability ofeach section of the final form of the PPT based on the data fromexaminees involved from the field testing. The number ofexaminees for each section is: Listening (96), Reading (110), andStructure (123).

Table 9Reliability of the

PPT by Section

Section reliabilityListening Comprehension .92Reading Comprehension .90Structure .88

The figures presented in Table 9 indicate that the PPT is ahighly reliable tests which can be used with confidence byprograms needing trustworthy measures of Polish languageproficiency.

6.2 Measurement Precision

Any measurement involves a degree of error. The higher theprecision of the measurement, the smaller the error. In ClassicalTest Theory, the degree of error is usually indicated by thestandard error of measurement (SEM). One of the limitations of

33

37

this approach is that it represents the SEM as being the same forall examinees. Test scores, however, are unequally precise forexaminees at different levels of ability. Item Response Theory(IRT), used in scoring the PPT, also provides a measure of theprecision of each score on the PPT Scale. In IRT, this measure iscalled the standard error of the ability estimate. Appendix J:Table of Standard Error of Estimation (also contained in the PPTTest Manual) contains the standard errors for PPT scores, roundedto the nearest whole number.

A traditional SEM may also be calculated for the PPT on thePPT Scale. These are presented in Table 10. Figures have beenrounded to the nearest whole number.

Table 10Standard Error of Measurement (SEM)

for the PPT(in Rounded Scale Score Units)

Section SEMListening 6Reading 7Structure 7

Both the SEM from Table 10 and the standard error of theestimate from Appendix J give an indication of how accurate asingle test score is. Either can be used to construct confidenceintervals around PPT scaled scores. The confidence interval takesmeasurement error into account. One way to view this confidenceinterval is to say that the examinee's true score (i.e., theaverage score an examinee would receive if it were possible togive him or her the test a very large number of times) would lie,with 67% certainty, within plus one or minus one standard errorof the given score. Another way to view this is to say that on are-test, if there has been no change in the examinee's trueproficiency, the examinee will score within plus or minus onestandard error of his or her score about 67% of the time.

A careful examination of Appendix J reveals that the most

34

precise area of measurement on the PPT lies between 66 and 136 onthe PPT Scale. For scores above 136, the measurement precision ofthe test rapidly diminishes. A score of 136 indicates aperformance well above the cut-off point for the fourth quartilefor first and second year students, and in the third quartile forstudents in the category "third year and above." In other words,the measurement precision of the PPT should not be of concern ininterpreting the scores of first and second year Polish studentson any section of the test. However, test score users should beaware that the PPT may not be able to discriminate accuratelybetween very high scoring examinees on the PPT; i.e., examineeswho score above 135 on any section. Table 8 indicates that suchexaminees would generally be individuals who have completed threeor more years of Polish study and who speak Polish at home.

6.3 Validity of the Test

Validity refers to the extent to which a test actuallymeasures what it purports to measure. The PPT claims to measurean examinee's proficiency in understanding authentic spoken andwritten Polish, and ability to deal with Polish structure.

The validity of any test cannot be "proven;" it can only beestablished by the collection of evidence that the test is indeedmeasuring what it purports to measure. Some commonly acceptedtypes of evidence include evidence for content, concurrent, andconstruct validity. Each type of validity, and evidence whichsupports it in the case of the and the PPT, are explained below.

Content Validity

Content validity is based on test content and refers to theextent to which the items in a test adequately sample thelinguistic and behavioral domains of the construct that the testis meant to measure. The PPT is intended to be a measure ofproficiency in dealing with every-day "real-life" Polish.Validity based on content thus entails an examination of thedegree to which the tests, in their stimulus passages, sample

35

J

from the corresponding language-use situations the examinee mightbe expected to encounter in real life.

1. Content Domains

The ACTFL Guidelines for listening and reading postulatecertain content and functional domains as appropriate to theproficiency of an English-speaking learner of Polish at differentlevels of proficiency. The developers of the PPT used authenticlistening and reading passages and wrote corresponding items toreflect these domains. Content and functional domains can begrouped into three categories, which represent increasing levelsof proficiency in Polish as a second language. These levels maybe designated as:

Intermediate Level-- Survival. At this level, the learner can"survive" by performing simple daily tasks such as shoppingfor food, clothes, and incidental necessities. PPT listeningcomprehension items at this level require the understandingof oral language typically heard in locations such as agrocery store, train station, bank, bus, and airport. PPTreading comprehension items at this level include certainsigns, advertisements, storefront designations, simplemessages with a social purpose, and short straightforwardinstructions encountered in daily life.

Advanced Level--Tourist Resident in Polish Society. At thislevel, the learner can understand social conversation oneveryday topics and can read short, descriptive narrations.PPT, listening comprehension items at this level requirecomprehension of short conversations between two peopledealing with general, every day topics, or the comprehensionof passages such as news items and reports dealing primarilywith factual information. PPT reading comprehension items atthis level require the examinee to comprehend main ideas andsome facts from passages excerpted from newspaper articles,public notices, personal correspondence, and popularstories.

Superior Level-- Foreign Professional in Polish Society. Atthis level, the learner can comprehend oral and writtenlanguage on virtually any matter of general interest toeducated native speakers in authentic Polish situations. PPTlistening stimuli at this level involve conversationsbetween two speakers dealing with topics of a generalacademic nature such as recent advances in health care and

36

current events, or longer discourse excerpted from speechesor reports. Questions may deal with the speaker's opinion orattitude, or they may require inference on the part of theexaminee. Reading stimuli at this level are generally longerand more technical and are followed by a questions dealingwith basic factual information covered in the passage, orinference or analogy based on information presented in thepassage. The passages cover a wide variety of daily topicssuch as those that may be found in a newspaper or magazine,e.g., results of a public opinion survey, job descriptions,and vacation travel.

For the Structure section, the PPT tests knowledge of

corre Polish syntax. Unlike listening or reading comprehension,

the g;.ructure section does not involve real-life language-use

1.Asks per se, though clearly knowledge of Polish syntax is a part

of comprehending spoken and written Polish. Items testing

recognin of Polish syntax on the PPT are of two types. The

first tyl. lasts knowledge of correct structure by requiring the

examinee to choose from among four options the correct word or

phis:.; that should be inserted in a blank within a printed

paragraph. The second type tests structural knowledge by

re.csuiring the examinee to find from among four underlined words

the one that is grammatically incorrect. In this section of the

PPT, only Polish is used fo: both the stimulus and response

options.

Table 2 of this report contains a list of the topic areas

covered in the PPT listening and reading comprehension sections.

Table 3 contains a list of speaking functions covered on the

listening section of the test. A list of the comprehension tasks

on the listening and reading comprehension sections is contained

in Appendix K: Comprehension Tasks for the PPT Items. A list of

grammar topics (by part of speech) covered in the structure

section can be found in Appendix L: Grammatical Areas Covered inthe PPT Structure Items.

2. Authentic Sources

Stimulus passages for listening, reading and structure on

37

41

the PPT were drawn from authentic language sources. Itemdevelopers collected passages for the reading comprehension andstructure sections from sources such as Polish language

newspapers, magazines, journals, books, street signs, schedules,and tickets. For stimulus passages for listening comprehension,they listened to and transcribed listening passages from sourcessuch as Polish language news broadcasts, movies, andannouncements. These texts were modified only to the extent thatthey needed to be clarified when taken out of the larger context.Questions were designed to check comprehension of the meaning ofpassages, and, for more difficult passages on the PPT, to checkcomprehension of opinions, attitudes or inferences contained inthe passages. A representative list of sources for the listeningand reading passages used on the PPT is contained in Appendix M:Sources for PPT Passages.

In terms of content validity, then, it can be demonstratedthat the PPT items have been drawn from real-life use of Polishlanguage.

Concurrent Validity

A second type of evidence of validity is concurrentvalidity. Concurrent validity refers to the extent to which atest score correlates with results that may be obtained throughthe use of independent criteria external to the test, measured atthe same point in time, to see if expected relationships exist.

For the PPT, the reported level of Polish study may be usedas an external criteria. It would be expected that scores on thetests would increase as amount of study increases. Both Tables 7and 8 indicate that this is the case for the PPT. In each skill,the median (Table 7) and average (Table 8) score increases as thelevel of Polish instruction increases. These tables thus provideevidence of the concurrent validity of the PPT as a measure ofPolish language proficiency.

38

Construct Validity

A third way of examining validity is construct validity. Thegoal of construct validity is to determine whether or not a testmeasures a single underlying trait. One assumption of the IRTmodel used to develop and score the PPT is that the items are"unidimensionals; that is, only one examinee ability or trait isnecessary to account for performance on the test. The fitstatistics provided by the IRT analysis provide evidence of theextent to which unidimensionality exists. Although perfect fit isimpossible, it may be argued that if the overwhelming majority ofitems are appropriately fitting, then there is strong evidencefor the construct validity of the test. For each item, astandardized outfit statistic is calculated. One common criterionis to consider any item for which this statistic is above 2.00 orbelow -2.00 as misfitting. Using this criterion, only 3 of the 50listening items (6%) and 2 of the 50 reading items (4%) on thefinal form were misfitting. This high degree of fit providesstrong evidence for the construct validity of these measures. Inthe structure section, 5 of the 35 items (14%) were misfitting.Although 14% represents a higher proportion of misfitting itemsthan for the listening and reading sections, the figure is lowenough to lend some support the construct validity for thestructure section. It may be noted that one of these fivemisfitting items has been revised in the final form.

Validity Summary

The above discussion provides evidence for the validity ofthe PPT. CAL plans to continue research pertinent to the validityof the test and will update the validity information section inthe PPT Test Interpretation Manual as appropriate.

6.4 Intercorrelations Among Test SubscoresThe three sections of the PPT are designed to measure

different skills within the general domain of Polish proficiency.It is expected that these skills are interrelated; i.e., persons

39

43

who are highly proficient in one skill area will tend to beproficient in the other skills as well. However, theintercorrelations are not expected to be perfect. If they were,there would be no need to report scores for each section; thesubscores would represent the same rather than different aspectsof language proficiency.

Table 11 shows the Pearson product-moment correlationcoefficients measuring the extent of relationships among thethree subsections of the PPT based on scores on the final form ofthe PPT for the 86 examinees who took all three sections in thefield testing.

Table 11Intercorrelations Among Subscores

List ReadListeningReading .84Structure .82 .92 OW MM. AM SIND

Table 11 indicates that the is a fairly strong relationshipamong the skills tested by the three subsections of the test.This relationship is strongest between reading and structure(.92), which may be expected in a highly inflected language asPolish. Nevertheless, it is reasonable to say, that eachsubsection scores provides some unique information about theexaminee's proficiency in the Polish language. This pattern ofintercorrelations between the test scores helps support thevalidity of the constructs the PPT claims to measure.

40

7. Operationalization of the PPT

Throughout the project, CAL staff sought to have the greatestamount of input possible from the Polish language teachingcommunity and to actively disseminate information about theproject. Towards the end of the project, the project staffcreated all the necessary materials to ensure that the PPT couldbe appropriately operationalized; i.e., that any appropriateindividual may administer the test to examinees in his or herprogram in a smooth and timely manner, that the testadministrator would have information on how to interpret testscores, and that steps to safeguard the security of the testwould be in place. These materials were described in Section1.5. This chapter describes some additional steps to disseminateinformation on the PPT program.

7.1 Information Dissemination During the ProjectDuring the test development phase, efforts were made to inform

the Polish language teaching community of the project, and inmany cases allowed for their input. A general announcement onthe project was developed in response to requests forinformation. A copy this announcement is supplied as Appendix N:PPT Announcement. In addition, CAL staff made several conferencepresentations. An outline of dissemination activities to datefollows.

h. The Modern Language Journal

An announcement about the project was sent to the Newsand Notes Editor of the Modern Language Journal forpublication.

B. American Association of Slavic and_Iast EuropeanLanguages

Mr. Xixiang Jiang, the Project Coordinator, delivered apresentation on the PPT development project at theAATSEEL annual conference in San Francisco, December28-30, 1991.

41

Mr. Jiang also presented on the completed project atthe annual AATSEEL conference in New York on December28, 1992.

C. George Washington University Colloquium on Forgignlanguage Testing

Project staff Dorry Mann Kenyon, Rafal Kwiatkowski, andCharles W. Stansfield presented an hour presentation onthe PPT entitled "Testing reading and listeningproficiency in Polish" on April 10, 1992, at GeorgeWashington University in Washington, DC.

D. 50th Anniversary International Congress of the PolishInstitute of Arts and Sciences of America

7.2

Mr. Xixiang Jiang, delivered a presentation on PPTProject in a session entitled "Teaching Polish Languageand Culture" at the annual conference in New Haven,Connecticut, June 18-20, 1992.

qu a t I w t ect el t nSeveral steps will be taken to disseminate information about

the PPT upon project completion, including the following. AnInformational Flyer will be developed and mailed to Polishprograms across the country and to members of the AmericanAssociation of Teachers of Slavic and East European Languages.In addition, copies of this final report will be made availableto the public through the ERIC Clearinghouse. Finally, CAL staffwill be available to present on the project at relevant venues.

42

46'

Appendix A

PPT Test Booklet

47

POLISH PROFICIENCY TEST

General Directions

This is a test of your ability to understand spoken and written Polish. There are threesections in this test: Listening Comprehension, Reading Comprehension and Structure.

In the Listening Comprehension section, you will hear and answer questions about anumber of spoken passages in Polish, including single sentences, conversations, radiobroadcasts, and other types of passages. In the Reading Comprehension section, you willread and answer questions about a variety of printed materials ranging from one or twowords to texts of paragraph length. In the Structure section, you will be asked to completethe missing portion of paragraphs written in Polish, and identify Polish words withinparagraphs that must be changed in order for the text to be correct. The test is intentionallychallenging, and you are not expected to know the answer to every question.

In each of the three sections of the test, the questions are printed in the test booklet. Youare to mark your answer to each question on the separate answer sheet provided. Make nomarks on the test booklet itself.

Your score on the test will be based on the number of questions you answer correctly.Throughout the test, it will be to your advantage to answer every question, even thoughyou may not be sure your answer is correct. There is no penalty for guessing.

When marking your answers, be sure that your mark is dark and completely fills theintended answer space. If you make a mistake or wish to change an answer, erase yourfirst answer completely. Questions for which more than one answer is marked will receiveno credit.

The Listening Comprehension and Structure sections are each divided into several parts.Additional directions are given at the beginning of each part.

DO NOT OPEN THE TEST BOOKLET UNTIL YOU ARE TOLD TO DO SO.

Appendix B

PVT Answer Sheet

49

SE

E IN

ST

RU

CT

ION

S O

N S

IDE

2

LAS

T N

AM

EF

IRS

T N

AM

EM

I

oot000000000000000A0000

esk000eoeoeeeeeeeoeopee

e' (D00000@goe®.eoe@meAa@e

e @e@o@o@seeelcUcic);

oo

000e0000000plocft o oo

o00000000oo0000 0 eo

Ve@e@o@o@cp@pl&ev

c)o

eeeeeeeeeee4eole

eo

00000000000Oodo

poo

0000o000oao000100lo

,00

(101000000,0000000A0

A0

000000000605000000

000888000g800,000

,eo

e4e00000eae*ee04 epee

i@o@wpo@@@@eo@eso cy (De

eoeoeoeeepopeoe 0 00

ae@e@e@ecpe@plesi© o (De

e eeeevogo%00@el® e c)0

e a@osaloi@o.c>p000mioelo o0

o ol000ao0000010,0.®00 oo

e eleeekleeese ege44 eo

e eeeeeeeeeo eaeole,eo

o eeeeeeeeise'oeoeaoeoe

o eosp00000000lorgooleelee

O 0 0 0 0 e oeeoeoe0000elooe&o'oee

O00000000000YI(DOC)00V00

TE

ST

DA

TE

SO

CIA

L S

EC

UR

ITY

NU

MB

ER

BIR

TH

YE

AR

CE

NT

ER

CO

DE

TE

ST

FO

RM

MO

.D

AY

YR

.

Jeri.

0Fe

b.o

mar.00eee@eeeee.oeeoeaeoee

AP,000000000000000100000

mw0100(:),0001000.0()0)

A.,000100(D@00,&00VC)010010000

Jac)

AL ,lo

swo

w-0

wv.0

D...o

eoeeeoaor00000le00.(D0

mcpecpooav000cp0o0oocIe

ocpocpcp@@ooe0oo'.ae@e

oot0000000:o.oxpopeozoo

eoo(DocpocK)ea@aockao

oeoeeeee'laogeOleacyeoe

@0

0

I II

I15

0i1-1-1I1141

POLISH PROFICIENCY TEST

Cen

ter

for

App

lied

Ling

uist

ics

1. W

hat i

s yo

ur s

ex?

()M

ale

0 F

emal

e2.

Wha

t lan

guag

e is

mos

t fre

quen

tly u

sed

in y

our

hom

e?

0 E

nglis

h0

Pol

ish

0 O

ther

(sp

ecify

)

3.W

hich

of t

he fo

llow

ing

mos

t app

ropr

iate

ly d

escr

ibes

the

leve

l of y

our

CU

RR

EN

Tor

MO

ST

RE

CE

NT

Pol

ish

lang

uage

cla

ss?

0 T

hird

Yea

r H

igh

Sch

ool

0 F

ourt

h Y

ear

Hig

h S

choo

l

0 F

irst S

emes

ter

Col

lege

(1s

t Yea

r P

oris

h)

0 S

econ

d S

emes

ter

Col

lege

(1s

t Yea

r P

olis

h)

0 T

hird

Sem

este

r C

olle

ge (

2nd

Yea

r P

olW

,)

0 F

ourt

h S

emes

ter

Col

lege

(2n

d Y

ear

Po

h)

0 F

ifth

Sem

este

r C

olle

ge (

3rd

Yea

r P

olis

h,

0 S

ixth

Sem

este

r C

olle

ge (

3rd

Yea

r P

olis

h)

0 S

even

th S

emes

ter

Col

lege

or

Abo

ve (

4th

Yea

r P

olis

h or

Hig

her)

0 O

ther

(sp

ecify

)

SID

E 1

AB

CD

1000

®A

BC

D20

00®

AB

CD

300'

100

AB

CD

<X

XX

)A

BC

D80

000

AB

CD

800®

®A

BC

D70

®®

®A

BC

D80

0100

AB

CD

800®

®A

BC

D10

0000

1 I

1 1

1 1

1111

111

AB

CD

7100

00A

BC

D12

0000

AB

CD

1300

00A

BC

D14

0000

AB

CD

1500

00A

BC

D18

0000

AB

CD

1700

00A

BC

D18

000®

AB

CD

1800

0®A

BC

D20

0000

1111

11

AB

CD

210000

AB

CD

2200

00A

BC

D23

0000

AB

CD

2400

00A

BC

D25

0000

AB

CD

2600

00A

BC

D27

0000

AB

CD

2000

00A

BC

D29

0000

A 8

C D

3000

00

AB

CD

3100

00A

BC

D32

0000

AB

CD

3300

00A

BC

D34

0000

AB

CD

3500

00A

BC

D36

0 0

00

AB

CD

3700

00A

BC

D38

0000

AB

CD

3800

00A

BC

D40

0000

1111

11

4100

00A

BC

D42

0000

AB

CD

4300

00A

BC

D44

0000

AB

CD

4500

00A

BC

D46

0000

AB

CD

4700

00A

BC

D48

0000

AB

CD

4900

00A

BC

D

5000

0t1

1111

11

IS

IDE

2

AB

CD

AB

CD

AB

CD

AB

CD

AB

CD

5100

0e61

0000

7100

0081

0000

9100

00A

BC

DA

BC

DA

BC

DA

BC

DA

BC

D52

0000

6200

0012

0000

8200

0092

0000

AB

CD

AB

CD

AB

CD

AB

CD

AB

CD

5300

0063

0000

1300

0083

0000

8300

00A

BC

DA

BC

DA

BC

DA

BC

DA

BC

D54

0000

6400

0074

0000

8400

0®84

0000

AB

CD

AB

CD

AB

CD

AB

CD

AB

CD

5500

0065

000®

7500

®0

8500

0095

00®

0A

BC

DA

BC

DA

BC

DA

BC

DA

BC

D56

0000

6600

0016

0000

8600

0096

0000

AB

CD

AB

CD

AB

CD

A B

C D

AB

CD

5700

0067

0000

1100

0091

0000

9100

00A

BC

DA

BC

DA

BC

DA

BC

DA

BC

D58

0000

6800

0078

0000

8800

0®98

0000

AB

CD

AB

CD

AB

CD

AB

CD

AB

CD

5900

0069

0000

1900

0089

0000

9900

00A

BC

DA

BC

DA

BC

DA

BC

DA

B C

6000

0010

00®

®80

0000

9000

®®

1000

00)

AB

CD

1010

000

AB

CD

1020

000

AB

CD

1030

000

AB

CD

1040

000

AB

CD

1°60

000

AB

CD

1060

000

AB

CD

1137

00®

0A

BC

D10

8000

0A

BC

D10

9000

0A

BC

D11

0000

0

Tra

n-(5

2by

NC

S

AB

CD

1110

000

AB

CD

1120

000

AB

CD

1130

00®

AB

CD

1140

000

AB

CD

1150

000

AB

CD

1160

000

AB

CD

1170

00®

AB

CD

1180

000

AB

CD

1190

000

AB

CD

1200

000

EP

-472

35.3

21A

9400

1111

1111

AB

CD

1210

000

AB

CD

1220

000

AB

CD

1230

000

AB

CD

1240

000

AB

CD

1250

000

AB

CD

1260

000

AB

CD

1270

000

AB

CD

1280

000

AB

CD

1290

000

AB

CD

1300

000

Prin

ted

in U

.S.A

.

II111I

AB

CD

1310

000

AB

CD

1320

000

AB

CD

1330

000

AB

CD

1340

000

AB

CD

1360

000

1977

by

Nat

iona

l Com

pute

r S

yste

ms.

Inc.

PO

LIS

H P

RO

FIC

IEN

CY

TE

ST

Cen

ter

for

App

lied

Ling

uist

ics

EX

AM

PLE

S

WR

ON

G

1 0

®0

0W

RO

NG

2 0

0'0

WR

ON

G

3 0

Go®

RIG

HT

40

®

IMP

OR

TA

NT

DIR

EC

TIO

NS

FO

R M

AR

KIN

G A

NS

WE

RS

Use

*2

penc

il on

ly.

Do

NO

T u

se in

k or

bal

lpoi

nt p

ens.

Mak

e he

avy

blac

k m

arks

that

fill

the

circ

leco

mpl

etel

y.

Era

se c

lean

ly a

ny a

nsw

er y

ou w

ish

to c

hang

e.

Mak

e no

str

ay m

arks

on

the

answ

er s

heet

.

Li SIG

NA

TU

RE

TE

ST

BO

OK

LET

NU

MB

ER

1 11

111

1111

1111

I 1

I I

IIII

1 I

I I

Appendix C

PPT Supervisor's Manual

54

PPTPOLISH PROFICIENCY TEST

SUPERVISOR'S MANUAL

October 1992

Copynght (c) 1992 by Center for Applied Linguistics, Washington, DC. All rights reser\ ed.

TABLE OF CONTENTSpage

IMPORTANT INFORMATION ON TEST ADMINISTRATION 1

Test Security 1

Communication with CALLength of AdministrationOrder of Administration 2Test Center Codes 2

PRIOR TO THE TESTING DATE 3Receipt of Test Materials 3Checking the Conditions of Test Materials 3Distribution of Examinee Handbooks 3Arrangements for a Testing Site 3

ON THE TESTING DATE 4Materials Needed for Test Administration 4Admission to the Testing Room 4Seating the Examinees 4Prohibited Materials 4

ADMINISTERING THE TEST 5Distributing Test Materials 5Filling Out the Answer Sheet 5One-day Administration 7

With Structure Section 7Without Structure Section 7

Two-day Administration 11

With Structure Section 11

Without Structure Section 11

AFTER THE TEST 16Checking Collected Materials 16Returning Test Materials to CAL 16

IMPORTANT INFORMATION ON TEST ADMINISTRATION

Test Security

It is extremely important that the PPT be safeguarded and administered under secure conditions ateach participating institution. By agreeing to serve as the supervisor of the test, you assume theresponsibility of enforcing test security from the time you receive the test materials until you returnthem to CAL. It is, therefore, requested that you adhere to the following conditions:

I. Keep all test materials either in your immediate physical possession or in a locked cabinet orother secure area under your control.

2. Do not copy, of allow others to copy, any portion of the test booklet or tape, or make any notesor transcriptions of the test booklet or tape cement.

3. Allow only those individuals who are to be tested to see the test materials, and only at the timeof test administration and under the specific procedures described in these instructions.

4. Should any irregularities occur, report them in the Supervisor's Report Form included in the testPackage.

Your serving as test supervisor constitutes an implicit agreement to conform with the aboveconditions. Your cooperation in this important matter is greatly appreciated.

Communication with CAL

If it is necessary to contact CAL by telephone, please call (202) 429-9292 and ask for the PPTcoordinator. Correspondence by mail should be addressed to:

Coordinator, Polish Proficiency TestCenter for Applied Linguistics1118 22nd Street, NWWashington, DC 20037FAX: (202) 659-5641

Length of Administration

The PPT may be administered in either a one-day session lasting approximately 2 1/2 hours or intwo sessions lasting about 1 1/2 hours on the first day and an hour on the second day. The times foreach section is as follows: about 40 minutes for the Listening Comprehension Section, 50 minutes forthe Reading Comprehension Section and 30 minutes for the optional Structure Section. A period of 10minutes is also needed for examinees to fill in the Personal Information section of the machine-scoreableanswer sheet. Please note that it is extremely important that this background information is completedcorrectly.

Programs placing a high emphasis on developing proficiency and/or decreased importance ongrammar may decide to omit the Structure section of the PPT. All scores and norming tables are basedsolely on section scores, so that ommitting the Structure section will in no way hinder theinterpretability or usefulness of test scores. If the Structure section is omitted, the required testadministration time is decreased by about 30 minutes.

Order of Administration

In a one-day administration, the PPT will be administered in the order of the PPT test booklet:Listening. Reading and Structure (if included). In a two-day administration, the Listening and Structure(if included) sections are administered on the first day, with the Reading section administered on thesecond. These options may be diagrammed as followed:

One-day Administration: Listening -- Reading (Structure)

Two-day Administration: Day One: Listening (Structure)Day Two: Reading

For the two-day administration, supervisors must collect and store test booklets and answer sheetsafter the first day and redistribute them on the second.

Test Center Codes

Each participating institution is assigned a 3-digit test center code for data processing purposes. Youcan find your test center code on the top of the Test Materials Checklist enclosed in the package. Youwill need this information when instructing the examinees to fill out their answer sheets.

2

PRIOR TO THE TESTING DATE

Receipt of Test Materials

Immediately after you receive the shipment of test materials, verify the contents against the yellowTest Materials Checklist. If you find any discrepancy, you should contact CAL immediately forreplacement of materials.

After receiving the test materials, be sure to place them in a securely locked area. Do not allowanyone but your designated associate or a responsible official to have access to the storage area.

Checking the Conditions of Test Materials

Although every effort is made to ensure that test materials are in good condition before they areshipped, it is still possible that some defects may escape our attention or may occur during mailing. Itis therefore important that you check the condition of the test booklets and test tapes. The followingsteps are recommended:

1. Check to make sure that all test booklets are properly sealed.2. Spot-check the test tapes by listening to random segments all the way to the end to make sure

that there are no noticeable problems.

Distribution of Examinee Handbooks

In order to prepare the examinees for taking the PPT, it is important that you give each examineea copy of the Examinee Handbook before the testing date. The Examinee Handbook providesinformation on the test and supplies some sample and practice test items. You should receive enoughcopies for your examinees with your order for the test. You may photocopy the Examinee Handbookfor other interested individuals if you wish, but all other test materials must be kept confidential.

Arrangements for a Testin Site

You should locate a testing site that is comfortable and free from distraction. The listeningcomponent of the PPT requires a quiet room with good acoustics throughout and a high-quality cassetteplayback unit. The small speakers of personal battery-powered cassette recorders are not satisfactoryfor this purpose. The test can be administered in a language laboratory if there is enough room in theindividual booths for the students to place their test booklets and answer sheets flat in front of them.The testing room should be large enough so that examinees can be seated with three feet of space in alldirections between all examinees. Make sure that you will have access to the testing site for the TOTALAMOUNT of time you need it.

3

BEST COPY AVAILABLE

ON ME TESTING DATE

Materials Needed for Test Administration

In addition to the test materials sent to you by CAL, you will need the following items foradministering the test:

1. Playback equipment for the Listening Comprehension section. Check your playback equipmentto make sure that it is functioning properly. Adjust the volume control so that everybody in theroom can hear the recording clearly. If the playback unit has a tone control, it should be set tothe middle ('flat response") position or adjusted somewhat toward the treble. It should not beturned toward the bass position. Make sure that the tape is completely rewound afterwards. Twoidentical tapes are provided; you can use either one.

2. A timer, wristwatch, or other timepiece capable of being reset during the administration fortiming the Reading Comprehension section. Check to make sure that it is functioning properlyand has been completely reset to zero (or 12:00). There should always be at least two timepiecesin the testing room as a check against mistiming.

3. A few medium-soft (No. 2) black lead pencils with erasers. Examinees will have been asked tobring their own pencils, but some may have forgotten to do so. Normally, an amount of pencilsequivalent to 20% of the total number of examinees will be sufficient.

Admission to the Testing Room

About ten minutes before the testing starts you can start admitting the examinees into the testingroom. If an examinee shows up who did not register with you ahead of time, it is at your discretion todecide whether you wish to admit that examinee. Only those who have paid the S20 testing fee can beallowed to take the test.

All examinees must present proper identification (a photo ID such as a driver's license, student ID,or passport). You must verify that the person who registered for the test is indeed the same individualwho takes the test. If an examinee forgets to bring any identification, you should not admit him or herinto the testing room. There can be no exceptions to this rule, even if you think you know the examineevery well.

Seating the Examinees

When seating the examinees, direct them to seats at random to avoid any possibility of pre-arrangedcheating. They should not be allowed to select their own seats. If at all possible, allow at least one seatbetween any two examinees.

Prohibited Materials

While taking the test, examinees should have nothing on their desks but their pencils, test booklets.and answer sheets. The use of books, dictionaries, personal recording devices, photographic devices.pens, notes, and papers of any kind is prohibited during the test.

4

6 t)

BEST COPY AVAILABLE

ADMINISTERING THE TEST

Follow the procedures below when administering the test. ALL INSTRUCTIONS PRINTED INITALICS SHOULD BE READ VERBATIM. Do not depart from these directions unless notedotherwise.

Distributing Test Materials

After all examinees have been seated, start distributing the following materials:

1. Standard answer sheets2. Test booklets

Give the following instructions after the above materials have been distributed:

Please do not open your test booklet. For this test, you are going to give all youranswers on the answer sheet. Do not write anything in the test booklet. You must use aNo. 2 pencil for narking your answers. If you do not have one, please raise your hand.

Distribute pencils as necessary.

Filling Out the Answer Sheet

Instruct the examinees how to answer the questions by filling out the standardized answer sheet.(Depending on the examinees' degree of familiarity with standardized answer sheets, you may opt tosimplify the instructions, or you may want to give some examples on the board.)

This is a machine- scoreable test. You most mark your answers by completely filling inthe circle provided; otherwise the machine will not score your test correctly.

All your answers must be put on the answer sheet. Do not mark anything on the testbooklet; otherwise, your test score may be cancelled.

This answer sheet has a total of 135 items. The front page has 50 items and the backpage has the rest. [Hold up the answer sheet and flip over the two sides.]

Look at side one. in the space marked LAST NAME, print each letter of your last name, anddarken the corresponding circle below. [Pause]

In the space marked FIRST NAME, print each letter ofyour name, and darken the correspondingcircle below. [Pause]

the space marked MI, print the first letter ofyour middle name, and darken the correspondingcircle below. [Pause]

5

Li

In the space marked TEST DATE, darken the circle that says [give the current month], thenwrite today's day and year, and darken the circles below. [Pause]

In the space marked SOCIAL SECURITY NUMBER, write your social security number anddarken the circles below. If you do not have a social security number, please leave this spaceblank. [Pause]

In the space marked BIRTH YEAR, write the year you were born, and darken the circles below.[Pause]

In the space marked CENTER CODE, write the number [see the Test Materials Checklist foryour test center code] and darken the circles below. [Pause]

In the space marked TEST FORM, write '11' and darken the circles below. [Pause]

Next, on the top right hand side of the page, you are asked to answer a few questions aboutyourself and your Polish studies. This information will not affect your score in any way, nor willit be reported to anyone without eliminating your name. It is used to describe the group ofexaminees that takes the PPT. Please fill in this information now. [Wait until all have finished]

Now turn to side two. In the box labelled SIGNATURE, sign your name. In the box labelledTEST BOOKLET NUMBER, print the five-digit number printed on the top right corner of yourtest booklet. [Wait until all have finished]

Are there any questions? [Wait for response]

6

One-day Administration

With Structure Section

Administering theListening Comprehension Section

[NOTE: This section is always administeredFIRST]

Prepare the examinees to begin the test bysaying:

This test consists of three sections:Listening Comprehension, ReadingComprehension, and Structure. The totaltesting time is about two hours and tenminutes. There will be a short breakbetween the sections.

There are 50 questions in theListening Comprehension section ofthis test. Your answers should berecorded on Side I of the answersheet. Always make sure you markyour answer in the correct space onyour answer sheet.

We are now ready to begin the test.Please use the end of your pencil with theeraser to break the seal on the testbooklet, but do not open the booklet. TheListening Comprehension will beadministered by a tape recording. Allnecessary instructions are on therecording. If you cannot clearly hear therecording, please raise your hand.Otherwise I will not stop the recordinguntil it has come to the end. Anyquestions? [Wait for response]

Now I will start the tape.

Begin playing the test tape. The tape willstart with the words, This is the PolishProficiency Test." While these directions arebeing read, walk about the room to makesure that the tape is clearly audible to all

7

BEST COPY AVAILABLE 63

Without Structure Section

Administering theListening Comprehension Section

[NOTE: This section is always administeredFIRST)

Prepare the examinees to begin the test bysaying:

This test consists of two sections:Listening Comprehension and ReadingComprehension. The total testing time isabout an hour and a half. There will be ashort break between the sections.

There are 50 questions in theListening Comprehension section ofthis test. Your answers should berecorded on Side 1 of the answersheet. Always make sure you markyour answer in the correct space onyour answer sheer.

We are now ready to begin the test.Please use the end of your pencil withthe eraser to break the seal on the testbooklet, but do not open the booklet.The Listening Comprehension will beadministered by a tape recording. Allnecessary instructions are on therecording. If you cannot clearly hearthe recording, please raise your hand.Otherwise I will not stop the recordinguntil it has come to the end. Anyquestions? [Wait for response)

Now I will start the tape.

Begin playing the test tape. The tape willstart with the words, 'This is the PolishProficiency Test." While these directions arebeing read, walk about the room to makesure that the tape is clearly audible to all

examinees and make any necessaryadjustments. The total playing time isapproximately 40 minutes. Please do notleave the testing room during this time.During the playing of the test tape, it wouldbe advisable to walk quietly around the roomto make sure that the playback volume issatisfactory and that the examinees are fillingin their answer sheet properly.

At the end of the tape, the announcer willsay: "This is the end of the ListeningComprehension section. Please close yourtest booklet and wait for furtherinstructions.' You should now stop the tapeand say:

We will now have a short rest break.Please insert your answer sheet inyour test booklet and close yourbooklet. We will begin the next sectionof the test in three minutes. (You mayleave the room if you wish.]

Administering the Reading ComprehensionSection

Begin the Reading Comprehension section ofthe test by saying:

There are 50 questions in the ReadingComprehension section of this test. Youranswers should be recorded on Side 2 ofthe answer sheet, beginning with question51. Always make sure you mar* youranswer in the correct space on youranswer sheet.

We will now start the ReadingComprehension section. Please turn topage 9 in your test booklet and find'Section II Reading Comprehension.'There is a total working time of 50minutes for this section. You may beginworking on this section as soon as youhave finished reading the directions.

[START TIMER IMMEDIATELY]

examinees and make any necessaryadjustments. The total playing time isapproximately 40 minutes. Please do notleave the testing room during this time.During the playing of the test tape, it wouldbe advisable to walk quietly around the roomto make sure that the playback volume issatisfactory and that the examinees are fillingin their answer sheet properly.

At the end of the tape, the announcer willsay: 'This is the end of the ListeningComprehension section. Please close yourtest booklet and wait for furtherinstructions.' You should now stop the tapeand say:

We will now have a short rest break.Please insert your answer sheet inyour test booklet and close yourbooklet. We will begin the next sectionof the test in three minutes. (You mayleave the room if you wish.]

Adm.;_nistering the Reading ComprehensionSection

Begin the Reading Comprehension section ofthe test by saying:

There are 50 questions in the ReadingComprehension section of this test. Youranswers should be recorded on Side 2 ofthe answer sheet, beginning with question51. Always make sure you mark youranswer in the correct space on youranswer sheet.

We will now start the ReadingComprehension section. Please turn topage 9 in your test booklet and find'Section 11 Reading Comprehension.'There is a total working time of 50minutes for this section. You may beginworking on this section as soon as youhave finished reading the directions.

[START TIMER IMMEDIATELY]

8

64 BEST COPY AVAILA3LE

After 40 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 10 minutes remaining in thissection.

After 45 minutes, inform examinees:

There are 5 minutes remaining in thissection.

After 50 minutes, inform examinees:

This is the end of this section of the test.Please stop working now. We will nowhave a short rest break. Please insertyour answer sheet in your test bookletand close your booklet. We will begin thenext section of the test in three minutes.[You may leave the room if you wish.]

Administering the StructureSection

Begin the Structure section of the test bysaying:

There are 35 questions in the Structuresection of this rest. Your answers shouldbe recorded on Side 2 of the answersheet, beginning with question 101.Always make sure you mark your answerin the correct space on your answersheet.

We will now start the Structure section ofthis test. Please turn to page 25 in yourtest booklet and find 'Section IllStructure.' There is a total working timeof 30 minutes for this section. You maybegin working on this section as soon asyou have finished reading the directions.

(START TIMER IMMEDIATELY]

After 40 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 10 minutes remaining in thissection.

After 45 minutes, inform examinees:

There are 5 minutes remaining in thissection.

After 50 minutes, inform examinees:

This is the end of the test. Please stopworking now. Look over your answersheet carefully. Be sure all the marksyou made are dark and heavy. Eraseany marks on your test booklet.[Pause]

Please insert your answer sheer inyour test booklet and close yourbooklet. Do not leave the room untilyou are told to do so.

Finishing the Test

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, the examinees may bedismissed.

9

65 BEST COPY AVAILABLE

After 25 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 5 minutes remaining in thissection of the test.

After 30 minutes, inform examinees:

This is the end of test. Please stopworking now. Look over your answersheet carefully. Be sure all the marksyou made are dark and heavy. Eraseany marks on your test booklet.[Pause]

Please insert your answer sheet inyour test booklet and close yourbooklet. Do not leave this room untilyou are told to do so.

Finishing the Test

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, the examinees may bedismissed.

10

6i;

Two-day Administration

With Structure Section

DAY ONE

Administering the ListeningComprehension Section

[NOTE: This section is always administeredFIRST]

Prepare the examinees to begin the test bysaying:

This test consists of three sections:Listening Comprehension, ReadingComprehension, and Structure. Todayyou will take Listening ComprehensionSection and the Structure Section. Thrtotal testing time is about 70 minutes.There will be a short break betweenthe sections.

There are 50 questions in theListening Comprehension section ofthis test. Your answers should berecorded on Side 1 of the answersheet. Always make sure you markyour answer in the correct space onyour answer sheet.

We are now ready to begin the test.Please use the end of your pencil withthe eraser to break the seal on the testbooklet, but do not open the booklet.The Listening Comprehension will beadministered by a tape recording. Allnecessary instructions are on therecording. If you cannot clearly hearthe recording, please raise your hand.Otherwise I will not stop therecording until it has come to the end.Any questions? [Wait for response]

Now I will start the tape.

11

BEST COPY AVAILABLE

Without Stn aure Section

DAY ONE

Administering the ListeningComprehension Section

[NOTE: This section is always administeredFIRST]

Prepare the examinees to begin the test bysaying:

G

This test consists of two sections:Listening Comprehension and ReadingComprehension. Today you will takethe Listening section. The total testingtime is about 40 minutes.

There are 50 questions in theListening Comprehension section ofthis test. Your answers should berecorded on Side 1 of the answersheet. Always make sure you markyour answer in the correct space onyour answer sheet.

We are now ready to begin the test.Please use the end of your pencil withthe eraser to break the seal on the testbooklet, but do not open the booklet.The Listening Comprehension will beadministered by a tape recording. Allnecessary instructions are on therecording. If you cannot clearly hearthe recording, please raise your hand.Otherwise I will not stop the recordinguntil it has come to the end. Anyquestions? [Wait for response]

Now 1 will start the tape.

Begin playing the test tape. The tape willstart with the words, 'This is the PolishProficiency Test." While these directions arebeing read, walk about the room to makesure that the tape is clearly audible to allexaminees and make any necessaryadjustments. The total playing time isapproximately 40 minutes. Please do not

the testing room during this time.During the playing of the test tape, it wouldbe advisable to walk quietly around the roomto make sure that the playback volume issatisfactory and that the examinees are fillingin their answer sheet properly.

At the end of the tape, the announcer willsay: 'This is the end of the listeningComprehension section. Please close yourtest booklet and wait for furtherinstructions.' You should now stop the tapeand say:

We will now have a short rest break.Please insert your answer sheet inyour test booklet and close yourbooklet. We will begin the next sectionof the test in three minutes. [You mayleave this room if you wish.]

Administering the StructureSection

Begin the Structure section of the test bysaying:

There are 35 questions in theStructure section of this test. Youranswers should be recorded on Side 2of the answer sheet, beginning withquestion 101. Always make sure youmark your answer in the correct spaceon your answer sheet.

12

6

Begin playing the test tape. The tape willstart with the words, "This is the PolishProficiency Test.' While these directions arebeing read, walk about the room to makesure that the tape is clearly audible to allexaminees and make any necessaryadjustments. The total playing time isapproximately 40 minutes. Please do notleave the testing room during this time.During the playing of the test tape, it wouldbe advisable to walk quietly around the roomto make sure that the playback volume issatisfactory and that the examinees are fillingin their answer sheet properly.

At the end of the tape, the announcer willsay: "This is the end of the ListeningComprehension section. Please close yourtest booklet and wait for furtherinstructions." You should now stop the tapeand say:

This is the end of the test for the day.Please stop working now. Look overyour answer sheer carefully. Be sureall the marks you made are dark andheavy. Erase any marks on your testbooklet. [Pause]

Please insert your answer sheet inyour test booklet and close yourbooklet. Do not leave this room untilyou are told to do so.

Finishing the Test [Day One]

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, and when the examinees arereminded of the next testing date, theexaminees may be dismissed.

We will now start the Structuresection of this test. Please turn topage 25 in your test booklet and find'Section 111 - Structure.' There is atotal working time of 30 minutes forthis section. You may begin workingon this section as soon as you havefinished reading the directions.

[START TIMER IMMEDIATELY]

After 25 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 5 minutes remaining in thissection of the test.

After 30 minutes, inform examinees:

This is the end of the test. Please stopworking now. Look over your answersheet carefidt. Be sure all the marksyou made are dark and heavy. Eraseany marks on your lest booklet.[Pause]

Please insert your answer sheet inyour test booklet and close yourbooklet. Do not leave this room untilyou are told to do so.

Finishing the Test [Day One]

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, and when the examinees arereminded of the next testing date, theexaminees may be dismissed.

13

6

DAY TWO

After all examinees have been seated,distribute the standard answer sheets and testbooklets to the students, making sure theyeach receive their own copy.

Give the following instructions after thematerials have been distributed:

Please do not open your test booklet.For this test, you are going to give allyour answers on the answer sheet. Donot write anything on the test booklet.You must use a No.2 pencil formarking your answers. If you do nothave one, please raise your hand.

Distribute pencils as necessary.

Administering the Reading ComprehensionSection

Begin the Reading Comprehension section ofthe test by saying:

There are 50 questions in the ReadingComprehension section of this test.Your answers should be recorded onSide 2 of the answer sheet, beginningwith question 51. Always make sureyou mar* your answer in the correctspace on your answer sheet.

We will now start the ReadingComprehension section. Please turn topage 9 in your test booklet and find'Section II ReadingComprehension.* There is a totalworking time of 50 minutes for thissection. You may begin working onthis section as soon as you havefinished reading the directions.

[START TIMER IMMEDIATELY]

DAY TWO

After all examinees have been seated,distribute the standard answer sheets and testbooklets to the students, making sure theyeach receive their own copy.

Give the following instructions after thematerials have been distributed:

Please do not open your test booklet.For this test, you are going to give allyour answers on the answer sheet. Donot write anything on the test booklet.You must use a No.2 pencil forms..,:frking your answers. If you do nothaw one, please raise your hand.

Distribute pencils as necessary.

Administering the Reading ComprehensionSection

Begin the Reading Comprehension section ofthe test by saying:

There are 50 questions in the ReadingComprehension section of this test.Your answers should be recorded onSide 2 of the answer sheet, beginningwith question 51. Always make sureyou mark your answer in the correctspace on your answer sheet.

We will now start the ReadingComprehension section. Please turn topage 9 in your test booklet and find'Section 11 - ReadingComprehension.' There is a total

eking time of 50 minutes for thissection. You may begin working onthis section as soon as you havefinished reading the directions.

[START TIMER IMMEDIATELY]

14

7 6 BEST COPY AVAILABLE

After 40 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 10 minutes remaining inthis section.

After 45 minutes, inform examinees:

There are 5 minutes remaining in thissection.

After 50 minutes, inform examinees:

This is the end of this test. Please stopworking now. Look over your answersheet carefully. Be sure all the marksyou made are dart and heavy. Eraseany marks on your test booklet.[Pause]

Please insert your answer sheet inyour test booklet and close yourbooklet. Do not leave this room untilyou are told to do so.

Finishing the Test [Day Two]

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, the examinees may bedismissed.

15

7i

After 40 minutes, inform examinees of thetime remaining [Do NOT stop and reset thetimer]:

There are 10 minutes remaining in thissection.

After 45 minutes, inform examinees:

There are 5 minutes remaining in thissection.

After 50 minutes, inform examinees:

This is the end of this test. Please stopworking now. Look over your answersheet carefully. Be sure all the marksyou made are dark and heavy. Eraseany marks on your test booklet.[Pause]

Please insert your answer sheet inyour test booklet and close yourbooklet. Do not leave this room untilyou are told to do so.

Finishing the Test [Day Two]

Immediately collect the test booklet fromeach examinee, making sure that the answersheet has been inserted. Be sure to accountfor all test booklets distributed. When all testbooklets and answer sheets have beenaccounted for, the examinees may bedismissed.

AFTER THE TEST

Checking Collected Materiab

Immediately after you have finished administering the test, please do the following:

1. Completely rewind the test tape.

2. Separate the answer sheets from the test booklets.

3. Scan the answer sheets to make sure that all sections have been filled in properly.

4. Flip through the test booklets to make sure that there are no pencil marks or missing pages.Report any defective or missing test booklets or test tapes on the Supervisor's Report Form. Alsobe sure to include any other irregularities such as cheating (give the examinee's name),malfunctioning equipment, mistiming, absence of examinees due to illness, etc.

5. Compare the test booklet number on each examinee's test booklet against the number on sidetwo of the answer sheet to make sure they are the same.

Returning Test Materials to CAL

Within three days after the test is given, please return the materials requested on the Test MaterialsChecklist to CAL. Be sure to check off all materials being returned on the Test Materials Checklistand fill out and sign Supervisor's Report Form. Your comments and suggestions can be placed onthe back of the Supervisor's Report Form. These comments, regarding any aspect of the testincluding time allotment, clarity of the instructions and questions, etc., are all highly appreciated.

THANK YOU FOR YOUR ASSISTANCE!

16

Appendix D

PPT Examinee Handbook

7"

PPT

(POLISH PROFICIENCY TEST)

EXAMINEE HANDBOOK

Preliminary Edition

Fall, 1992

This handbook is for your use in preparing for the Polish Proficiency Test (PPT). Itcontains a description of the test, instructions for registering for the test, procedures tofollow on the day of the test, information about your scores, and sample test questions.Please familiarize yourself with the information in this handbook before taking the test andretain it for use in interpreting your test score.

Copyright © 1992 by Center for Applied Linguistics, Washington, DC. All rights reserved.

74

GENERAL INFORMATION

The Polish Proficiency Test (PPT) was developed by the Division of Foreign Language Educationand Testing at the Center for Applied Linguistics (CAL) in 1992 with the collaboration of Polishlanguage scholars and teachers in the United States and Canada. Funding for the test developmentproject was provided through a grant from the U.S. Department of Education.

The PPT is designed to evaluate the level of proficiency in Polish attained by American and otherEnglish-speaking learners of Polish. The test is designed to be most appropriate for students who havecompleted one or more years of Polish language study at the college level (or its equivalent).

Scores from the PPT can be used in making screening and placement decisions related to thefollowing:

admission to a Polish study programplacement within a Polish study programexemption from a Polish language requirementapplication for a scholarship or an appointmentcompetency testing upon exit from a Polish programcertification of Polish language proficiency for career placement

The PPT consists of 135 4-option multiple-choice items. Each examinee receives a test bookletcontaining the test questions and a separate, machine-scoreable answer sheet. All responses to themultiple-choice items are indicated on this sheet.

The PPT is divided into three sections: Listening Comprehension (50 items), ReadingComprehension (50 items) and Structure (35 items). In the Listening Comprehension section, you willhear listening passages consisting of utterances spoken by a single individual, excerpts fromconversations between two speakers, and longer single-speaker passages such as news broadcasts,advertisements, public interest announcements and so forth. Following each passage in Polish, you willbe asked a question in English for which you must choose the correct answer from among four optionsprinted in English in the test booklet.

Each item in the Reading Comprehension section will present you with a Polish text, followed bya question about the text and four possible responses, printed in English. This section is designed toassess your ability to understand a variety of printed texts ranging from a few words (e.g., street orstore-front signs, announcements) to passages of paragraph length (e.g., excerpts from newspaper ormagazine articles, personal and official correspondence, popular stories).

The Structure section has two parts. In the first, you must choose from among four options thecorrect word or phrase that is missing from a text. In the second, you must indicate from among fourunderlined words which one contains an error.

Questions to all items are printed in a single test booklet. Your answers will be marked on aseparate machine-scoreable answer sheet. Making any marks on the test booklet may result incancellation of your test score. The total testing time for the PPT is approximately two and a half hours.

1

75

1

Sample items from the PPT appear in the Appendix of this manual. Carefully study the directionsand the sample items. They will give you a good idea of the types of questions you will find on theactual PPT.

REGISTERING FOR THE TEST

The PPT is administered only through colleges or other institutions that offer a Polish studyprogram. There are two ways to register: institutional and individual registration. If your instructor orinstitution has arranged for you to take the test (institutional registration), your institution will informyou of the date, time, and location of testing and of any testing fees you may be charged.

If you wish to register for the test as an individual, you must first contact a local institution andregister there with a test supervisor, who will administer the test. The date and time you select to takethe test will be arranged by the supervisor at your mutual convenience. Your test supervisor may be aPolish instructor at your school. If you are not currently enrolled in any Polish language program, youmay contact the Coordinator of the Polish Proficiency Test at CAL (202-429-9292) and request thenames of institutions in your area at which you may be able to take the test.

When registering for the test, you will need to bring with you a photo identification card (driver'slicense, student ID, or passport). You will also be asked to give your name and social security numberto the test supervisor. The current testing fee for the PPT is $20, which you may pay to the supervisorby cash or check when you register or when you report to the testing center on the day of the test. Ifyou choose to pay by check, confirm with your supervisor whether the check should be made payableto the supervisor or to the participating institution. Do not write a check to CAL. Some institutionsmay cover the testing fee on your behalf, in which case you would not need to pay any fee.

PREPARING FOR THE TEST

As a test of general language proficiency, the PPT is not based on any particular textbook or courseof study, but on the kinds of language-use situations that are encountered in real-life contexts outsideof the academic setting. Because this type of proficiency can be achieved only through continued studyand practice with the language, it will probably not be helpful for you to study intensively any particulartextbook just prior to taking the test. The suggested preparation for the test is to carefully study the testdirections and sample questions in this handbook until you are thoroughly familiar with the testingprocedures and question types used in the test you will be taking.

On the day of the test, you must report to the testing center at least 10 minutes before the test isscheduled to begin; no one will be admitted after the test has begun. Remember to bring the followingitems with you:

1. a photo ID (driver's license, student ID, or passport);2. S20 testing fee (if it is not waived by your institution and you have not prepaid during

registration); and3. several sharpened medium-soft (No. 2) pencils with erasers.

If you forget to bring the first two items, you will not be admitted to the testing room. There canbe no exceptions to this rule.

2

7 0

The following materials are prohibited in the testing room: books, dictionaries, personal recordingdevices, photographic devices, pens, notes, and papers. Any cheating (giving or receiving assistance,taking the test for someone else, copying or recording any test material, etc.) will result in dismissalfrom the testing room and cancellation of your test scores. The testing fee will not be refunded undersuch circumstances.

REQUESTING SCORE REPORTS

After the test has been administered, the Center for Applied Linguistics will score the test and senda score report roster to the institution. You may then check with the institution for your test results. Thetime between CAL's receipt of the completed test materials and the sending out of the score roster isabout 2-3 weeks.

CAL also provides score reporting services to individual examinees for the purpose of officialcertification of their Polish language proficiency. This official score report can be sent either to you orto an institution you designate. You must use the Request Fonn for Official Score Reports on the lastpage of this handbook. Mail this form with $6.00 for each score report to be sent to the Coordinator,Polish Proficiency Test, Center for Applied Linguistics, 1118 22nd Street, N.W., Washington, DC20037. Payment may be made with a check drawn on a bank in the U.S. or by money order. Scorereports will be sent within 2 weeks following receipt of your request.

INTERPRETING YOUR TEST SCORES

The Meaning of the Score on the PPT Scale

Scores on the PPT for Listening, Reading and Structure are reported as scores on the PPT Scale.The scaled scores are based on your Polish language proficiency as demonstrated by the difficulty ofthe items you answered correctly, rather than only on the number of items you answered correctly.

Scores on the PPT Scale can be understood in several ways. First, a score on the PPT Scale canbe interpreted in a normative manner directly. This means that your scaled score can be compared tothe scores of a group of 70 examinees typical of North American students studying Polish at the collegelevel in any given year. Sixty-seven percent of this group were in their first year of Polish study, 24%were in their second, and 9% were in their third (or higher) year. Forty-eight percent were male and52% were female. Fifty-eight percent were of Polish ethnicity. Sixty-four percent of the group indicatedthat they spoke no Polish at home, while 3% spoke Polish at home "most of the time," 6% spoke Polishat home Ness than half the time," and 27% spoke Polish "very infrequently" at home.

A score of 100 on any section reflects the average (mean) performance of these 70 students on thePPT. If you scored 100 on any section, you have demonstrated average proficiency when compared tothis group. In statistical terminology, each 20 points on the PPT Scale is equal to a standard deviation.This means that approximately 95% of all scores from a group of examinees comparable to the normgroup will fall in the range from 60 to 140 on the PPT Scale.

'73;

Using the Table of Quartile Distributions

To better compare your scores with those of examinees in your year of study, use Table 1, whichprovides quartile scores on the PPT Scale for examinees in the first, second, or third (or more) yearof Polish language study. Quartiles represent the bottom, second, third and top quarter of the scoredistribution. If your score places you in the second quartile, this indicates that you performed in thesecond 25% of all students in your subgroup, and better than the bottom 25% of all the examinees inthat subgroup. A score in the fourth quartile indicates that you scored in the top 25% of all otherexaminees in the subgroup; i.e., better than 75% of all other examinees.

To use Table 1, first categorize yourself as either a first, second, or third (or more) year studentof Polish. Then find the category in the table that containsyour Listening, Reading or Structure sectionscore. Finally, look over in the left-most column to find the quartile corresponding to your score.

Table 1Quartile Distributions for the PPTby PPT Section and Year of Study

FFirst Year Second Year Third Year or Higher,

Quartile List Read Str List Read Str List Read Str

4th(76-100%)

99 andabove

105 andabove

106 andabove

121 andabove

119 andabove

123 andabove

139 andabove

144 andabove

154 andabove

3rd(51-75%)

92-98 93-104 93-105 105-120 107-118 110-122 130-138 127-143 122-153

2nd(26-50%)

83-91 8: -92 84-92 97-104 99-106 104-109 101-129 109-126 108-121

1st

(1-25%)82 andbelow

82 andbelow

83 andbelow

96 andbelow

98 andbelow

103 andbelow

100 andbelow

108 andbelow

107 andbelow

For example, if you are a student in a first year college-level Polish class and received a PPTScale score of 95 on the Listening Comprehension section, you would first look under thecolumn for first year students, then find the column for listening and under this column, find thescore group that contains your score. Your score of 95 is contained in the 92-98 range. Thiscorresponds in the left hand column to the third quartile. Thus, you know you have scored above50% of all first-year examinees in the Listening Comprehension section, but below the top 25%of students in first year Polish. Note that your PPT Scale score of 95, when compared with the70 examinees in the norming group, indicates you scored below their average score of 100.

Using the Group Means for Polish, non-Polish Speakers

A number of Polish larTnage students use Polish in the home. Such examinees may have anadvantage on a test of general language proficiency. However, since these examinees formedonly a small minority of the total number of students in the norming group, and since they docomprise a part of the total group of North American students studying Polish, they wereincluded in Table 1. Nonetheless, you may want to understand the meaning of your test scores

4

7F,

in terms of whether you speak Polish at home or not. Table 2 presents the average PPT Scalescore for each section by year of study for examinees who indicated that they do not generallyspeak Polish at home (labeled English) and for those who indicated that Polish is spoken abouthalf the time or more (labeled Polish). If no average is shown, there were not enough membersin the subgroup to calculate a reliable average score.

Table 2Mean Scores on the PPT Scaleby Level and Home Language

IListening Reading Structure

Year of Study English Polish English Polish English Polish

First Year 92 93 96 94 96

Second Year 107 -- 107 110 - --

Third Year (or higher) 124 -- 119 130 121 129

To use Table 2, find the subgroup to which you belong and compare your score to thatsubgroup's mean score. For example, if you were a third-year student who scored 125 on theReading Comprehension section and speak Polish at home, your score would place you slightlybelow the mean of 130 for your peers. On the other hand, if in the same situation you belongedto the "English" category, your score of 125 would be somewhat above the average of 119.

APPENDIX

Practice Questions for the

Polish Proficiency Test (PPT)

Directions:

In this Appendix, you will find the actual test directions followed by practice

items for each of the types of questions found in the PPT. Carefully work through

these items as if you were taking the actual test. Answers to these items appear at

the end of this section.

Remember: Your score on the test will be based only on questions you answer

correctly, so it will be to your advantage to answer every question, even though

you may not be sure that your answer is correct. There is no penalty for guessing.

6

Practice Questions for the PPT

Listening Comprehension Section

NOTE: All spoken material for this section is contained on a tape recording which will beplayed to you at the time of testing. For the practice questions below, the material printed initalics represents spoken passages that will not appear in print. Inside the small brackets, [W]indicates a woman's voice, [M] a man's voice, and [A] the English announcer's voice.

Part One

Directions: For each question in this part, you will hear a short spoken statement orquestion in Polish, followed by a spoken question about it in English. From the fourchoices, labeled A through D, printed in your test booklet, choose the one that correctlyanswers the question and mark the corresponding space on your answer sheet. Listen to thefollowing example:

[W] Mamusiu, Tatusiu, wychodze do kina z lcoletankami.

[A] With whom is the speaker going to the movies?

(A) Parents(B) Friends(C) Brothers(D) Sisters

Of the four choices, B, "Friends," is the correct answer to the question, "With whom isthe speaker going to the movies?" If this were one of the test questions, you would markthe space labeled B on your answer sheet.

For each question in this part and throughout the test, mark only one of the four spaces,A, B, C, or D.

1. [W] Prosimy do stolu.

[A] Where would this statement mostlikely be heard?

1. (A) Ina home(B) At a ticket counter(C) In a furniture store(D) In a classroom

2. [M] Mamy szaknie duto wypadkOw.Ludzie jetdta nieuwatnie.Wiekszok z tych wypadi5w, kthrezalatwiamy, nie powinna mieezupelnie miejsca.

2. [A] What is the most likely profession ofthis speaker?

781

(A)(B)(C)(D)

ScientistJournalistPolice officerTeacher

Practice Questions for the PPT

Part Two

Directions: For each question in this part, you will hear a short conversation in Polishbetween two speakers. The conversation will immediately be followed by a spoken questionabout it in English. From the four choices printed in your test booklet, choose the one thatcorrectly answers the question and mark the corresponding space on your answer sheet.Listen to the following example:

[W] Przepraszam bardzo, ile kosztuje to szynka?[M] 2009 ziotych kilo.[F] Poproszc 01 kilo.

[A] What does the woman want to know about the item?

(A) Its price(B) Its size(C) Its weight(D) Its quality

The correct answer to this sample item is A, "Its price."

3. [W] Tu SQ take warunki, to w tadnymnormalnym cywilizowanym kraju takietargowisko by nie istniaio. Cry tomotna zmienie, pana zdaniem?

[M] Tak, motna vnienie, preyodrobinie dobrej woliwlakicieli tego targowiska,jak i wiadz dzielnicy.

[A] According to the man, who needs tocooperate to solve the problems of themarket?

3. (A) The police and the owners of themarket

(B) The police and the inhabitants of thedistrict

(C) The owners of the market and thelocal authorities

(D) The inhabitants and the localauthorities

8

4. [M] Jak tam twoja Marysia uczy sic wszkok w tym roku?

[W] No wiesz, w tym roku poszla jut donowego liceum, ale okazalo sie, topoziom z matematyki jest o wielenitszy nit mygkli.fmy. Wybralis`my toliceum bo bylo blisko, ale wprzysziym roku zdecydujemy sie japrzeniege do liceum z rozszerzonymprogramem matematycznym.

[A] According to the woman, on whichbasis did she initially choose theschool for her daughter?

4. (A) Its location(B) Its program(C) Its teachers(D) Its reputation

Practice Questions for the PPT

Part Three

Directions: In this part, you will listen to a variety of spoken passages in Polish, includingradio broadcasts, announcements, and other types of material. After each passage, you willhear a spoken question about it in English. From the four choices printed in your testbooklet, choose the one that correctly answers the question and mark the correspondingspace on your answer sheet.

There is no example question for this part.

5. [W] A w naszym programie jut za chwilenowy film angielski.

[A] When would this televisionannouncement most likely be heard?

5. (A) Before a news program(B) During a sports event(C) Before a feature movie(D) After a documentary

6. [W] Do historii pizechodzi ismiejaca odponad czterdziestu lat lOdzkaOrkiestra Polskiego Radia i Telewizji.Pozostawia po sobie tysiace nagrarimagnetofonowych i plytowych.Orkiestre wyparly z anteny nagraniazagranicznej konkurencji.

983

[A] What is the main topic of this newsitem?

(A) The closing of a radio station(B) The anniversary of a record company(C) The disbanding of an orchestra(D) The reissuing of old tapes and

records

Practice Questions for the PPT

Reading Comprehension Section

Total Testing Time: 50 minutes

Directions: This section consists of printed Polish texts ranging in length from a fewwords to short paragraphs. Following each is a question in English. On the basis of theinformation given in the Polish text, choose the correct answer, (A), (B), (C), or (D), toeach question. Then mark the corresponding letter on your answer sheet next to the numberof the question.

There is no example question for this part. Begin work as soon as you have understoodthese directions. You will have 50 minutes to complete this section.

Premier na Dalekim Wschodzie

7. The article following this newspaperheadline most probably discusses

(A) an election(B) an official visit(C) a meeting with advisors(D) a political faux pas

Polska Iiczy obecnie ponad 32 milionymieszkaricOw i pod wzgledem liczebnotcizajmuje si6dme miejsce w Europie idziewietnaste w gwiecie.

8. According to this passage (written in1970), which position in the world doesPoland occupy in terms of population?

(A) Sixth(B) Seventh(C) Twelfth(D) Nineteenth

10

Pierwszym 50 Czytelnikom,ktOrzy jui w najblitszy pintek,6 wrzeinia, w godz. 15-16.odwiedzn nsszts trdakcje, funduje-my z okazji jubileuszu 45--lecia .Expressu Wieczomego'

bilety na sobotnio-niedzielnedyskoteki w studenckim klubie.STODOLA' przy ul. Ratorego

9. According to this contest announcement,what must readers do to win the prize?

(A) Participate in a dance(B) Write an essay(C) Visit an office(D) Purchase special tickets

Catkowicie lub czegclowoniewykorzystane bilety powinny byaprzedstawione w celu uzyskania zwrotuoptaty w okresie 30 dni od datywygagniqcia ich wainotci. Zwrotu zabilet dokonuje to biuro, w ktOrym zostaton wystawiony i optacony.

10. What procedure does this informationfound on an airline ticket explain?

(A) How to exchange the ticket(B) How to reconfirm reservations(C) How to get a refund(D) How to label luggage

8

Practice Questions for the PPT

Structure Section

Total Testing Time: 30 minutes

General Directions: The Structure section is composed of two parts. The total testing timefor the section is 30 minutes. Go directly to Part Two after you finish Part One.

Part One

Directions: In this part there are five Polish passages, each with five missing portions. Themissing portions are indicated by an underlined item number. Beneath each passage are fiveitems labeled with item numbers corresponding to those in the passage. For each item,choose one of the four options, (A), (B), (C), or (D), which completes the missing portionof the passage in a meaningful and grammatically correct way. Then mark thecorresponding letter on your answer sheet next to the number of the missing portion.

Study the following example:

29 bm. przypada 50 rocznica gmierci Ignacego Paderewskiego. Tego wlagnie dniaTeatr Wielki Ex I z premiera jedynej jego opery, ,Nanru". Libretto AlfredaNossiga oparte jest na Ex2 popularnej powiegci Jazefa Ignacego Kraszewskiego"Chats za wsia".

Jest to ostatnia premiera Teatru Wielkiego w tym sezonie, a jui w lipcu i sierpniu--przerwa urlopowa.

Ex 1. (A) wystcpowad(B) wystepuja(C) wystcpujc(D) wystepuje

Ex2. (A) motywy(B) motywach(C) motywami(D) motywow

For Ex 1, the correct answer is (D) and for Ex2, the correct answer is (B). These are theonly choices that complete the missing portions of the passage in a meaningful andgrammatically correct way. Begin work on this part as soon as you have understood thedirections.

Practice Questions for the PPT

W ciagu 11 lat tolnierze zawodowi i rezerwitci zobowiazani sa 12

ponownej przysiegi wojskowej, ktora bedzie sic kodczyla 13 .Tak mi dopomet

kg". Zolnierz sklaciajacy przysiege mote 14 od wypowiedzenia ostatniego zdania

roty - zadecydowal 15 czwartkowego posiedzenia Parlament.

11. (A) piee 13. (A) slow 15. (A) na czas(B) piatka (B) slowem (B) do(C) picciu (C) slowach (C) podczas(D) Piatka (D) slowami (D) od

12. (A) zloteniem 14. (A) przestapid(B) aby zlotye (B) odstapit(C) zlotony (C) przystapi4(D) do zlotenia (D) wstapid

Part Two

Directions: In this part, there are three Polish passages, each with five numbered lines.In each numbered line, there are four underlined words. These are marked (A), (B), (C),and (D). Find the ONE underlined word that must be changed in order for the line to begrammatically correct. Then find the number of the line on your answer sheet and markthe letter that corresponds to your answer.

Study the following example:

Od pierwszego lipca importerzy piwa i wina musza mied specjalne

Ex3. pozwolenia na przywoz. Pozwolenie taki mop dostad tylko osoby,(A) (B) (C) (D)

Ex4. ktore m..._ a licencje ministra rynku wewnetrznego na obrot(A) (B) (C) (D)

hurtowy napojami alkoholowymi. Ograniczenie jest tymczasowe i

obowiazuje do 31 grudnia bieiacego roku.

In Ex3, choice (B) should read "takie" in order to make the line grammatically correct.Therefore you should mark answer (B) on your answer sheet. In Ex4, choice (A) shouldread "maja" for the line to be grammatically correct. You should mark answer (A) on youranswer sheet. Begin this part of the test as soon as you have understood the directions.

12

Practice Questions for the PPT

- A czy z czasem pani obrazy zmieniaja sic?

16. Niewiele, dawniej byly podobni, mote jedynie troche spokojniejsze,(A) (B) (C) (D)

17. mole bardziej narracyjne, na pewnym bardziej slodkie. Ale trudno mi(A) (B) (C) (D)

18. o tym opowiadam. Ten sam problem mialam na obronie dyplomu.(A) (B) (C) (D)

19. Komisja pytala mnie, co mam dla powiedzenia.

(A) (B) (C) (D)

20. Powiedzialam im, te nic, widad natomiast, co mam do pokazywania.(A) (B) (C) (D)

Answers to Practice Questions

'OZ

D '61'81

9 'LI9 '91

D 'SIEl 'VICI 'CICI 'Z1D I I

D '01D '6CI '89 'LD '9

D S'V

D '£D 7V I

Polish Proficiency TestREQUEST FORM FOR OFFICIAL SCORE REPORTS

Print the required information below and enclose a check or money order for $6.00 made payable toPPT/Center for Applied Linguistics for each score report you request (i.e., $6.00 if requesting onescore report, $12.00 if requesting two). Detach and mail this form together with the correct paymentto: Coordinator, Polish Proficiency Test, Center for Applied Linguistics, 1118 22nd Street, N.W.,Washington, DC 20037.

Examinee Name:Last First

Social Security Number:

Test date: / (month / year)

MI

Testing Center:

Test Supervisor (if known):

Score Report 1 Score Report 2

Name of Score Recipient Name of Score Recipient

Institution (Testing Center) Institution (Testing Center)

Department Department

Number and Street Number and Street

City City

State/Zip Code State/Zip Code

Your signature Your signature

Date Date

* * * * * * * * *

Print the name and address of the institution to which you want a PPT official score report sent. Ifyou want the report sent to you, give your own name and address. Photocopy this form if more thantwo score reports are requested.

88

The Center for Applied Linguistics was chartered in 1959 as a non-profitcorporation in Washington, DC with initial support provided by the FordFoundation. While historically CAL has developed language tests to meet the needsof its various domestic and international projects, in 1986 CAL's language testinginitiatives were centralized in the Foreign Language Education and TestingDivision. The Division produces language tests for use in governmental,educational, and occupational settings. For information on CAL's test developmentcapabilities, contact Dr. Charles W. Stansfield, Director, Division of ForeignLanguage Education and Testing, CAL, 1118 22nd Street, NW, Washington, DC20037, (202) 429-9292.

Appendix E

PPT Test Interpretation Manual

90

POLISH PROFICIENCY TEST

TEST INTERPRETATION MANUAL

Preliminary Edition

Fall, 1992

Center for Applied Lingiustics1118 22nd Street, NW

Washington, DC 20037(202) 429-9292

PREFACE

The PPT Test Interpretation Manual has been prepared for individuals who are interested in thePolish Proficiency Test (PPT) offered by the Center for Applied Linguistics (CAL): forindividuals considering using the test, for those who administer the test, and for those who useor interpret scores attained on the test. It presents an overview of the testing program, the usesand purposes of the tests, information on the interpretation of test scores, and the statisticalcharacteristics of the test.

The Polish Proficiency Test was developed in the Division of Foreign Language Education andTesting at CAL through support provided by the International Research and Studies Program ofthe U.S. Department of Education. The test is the result of collaboration of the CAL withPolish language teaching professionals throughout North America. Below is a list of primaryindividuals who worked on developing the test.

Test Development Team

Charles W. StansfieldDorry Mann KenyonXixiang Jiang

Dosia Drycz-FreemanGrayzna KabatLisa M. PinsonRafal KwiatkowskiWaldek Walczynski

External Advisors

Waldemar MartyniukBarbara NiemczykLeonard PolakiewiczRobert RothsteinKrystyna SadlowskaAlexander SchenkerOscar SwannKrystyna WachowiczJoanna Williams

Center for Applied LinguisticsCenter for Applied LinguisticsCenter for Applied Linguistics

Center for Applied LinguisticsCenter for Applied LinguisticsCenter for Applied LinguisticsCenter for Applied LinguisticsCenter for Applied Linguistics

Stanford UniversityDickinson CollegeUniversity of MinnesotaUniversity of MassachusettsForeign Service InstituteYale UniversityUniversity of PittsburghDefense Language InstituteSUNY, Stony Brook

(Project Director)(Associate Project Director)(Project Coordinator)

(formerly)

(formerly)

This is the first edition of the PPT Test Interpretation Manual. Your suggestions for itsimprovement are most welcome. Please feel free to send any comments to the Coordinator,Polish Proficiency Test, Center for Applied Linguistics, 1118 22nd Street N.W., Washington,DC 20037.

The Polish Proficiency Test was developed by the Division of Foreign Language Education and Testing at the Centerfor Applied Linguistics under Grant Number P017A10051 from the International Research and Studies programof the U.S. Department of Education.

OILCopyright 1992 by Center for Applied Linguistics, Washington, DC.

All rights reserved.

92

GENERAL INFORMATION

History of the PPT

The Polish Proficiency Test (PPT) was developed by the Division of Foreign LanguageEducation and Testing at the Center for Applied Linguistics (CAL) in 1992 with thecollaboration of Polish language scholars and teachers in the United States and Canada. Fundingfor the test development project was provided through a grant from the U.S. Department ofEducation.

Test Content

The PPT is designed to measure the general proficiency in listening and reading attained byAmerican and other English-speaking learners of Polish. In keeping with its intended purposeas a measure of general language proficiency, the major focus of the test is on assessing theexaminee's ability to function in situations typical of real-life language-use situations. It wasdesigned for examinees at proficiency levels from Novice High to Superior according to theACTFL Proficiency Guidelines (American Council on the Teaching of Foreign Languages,1986). In terms of length of Polish study, the PPT has been designed to be appropriate forstudents who have completed one or more years of college level Polish language instruction. Thetest is suitable for students at the end of their first year of study at the college level. However,students who have completed less than one year of college-level study (or its equivalent) willmost likely find the test too challenging.

Test Format

The PPT consists of 135 4-option multiple-choice items. Each examinee receives a testbooklet containing the test questions and a separate, machine-scoreable answer sheet. Allresponses to the multiple-choice items are indicated on this sheet.

The PPT is divided into three sections: Listening Comprehension (50 items), ReadingComprehension (50 items) and Structure (35 items). In the Listening Comprehension section,examinees hear listening passages consisting of utterances spoken by a single individual, excerptsfrom conversations between two speakers, and longer single-speaker passages such as newsbroadcasts, advertisements, public interest announcements and so forth. Following each passagein Polish, examinees are asked a question in English for which they must choose the correctanswer from among four options printed in English in the test booklet. All listening passageshave been excerpted from authentic sources. To ensure the quality of the recording, the passageshave been rerecorded. All Polish voices on the cassette are native Polish speakers, carefullyselected for voice quality and authenticity of expression. The spoken material is delivered withnatural pacing and intonation.

Each item in the Reading Comprehension section presents examinees with a Polish text,followed by a question about the text and four possible responses, printed in English. Thissection is designed to assess the examinees' ability to understand a variety of printed texts

1

ranging from a few words (e.g., street or store-front signs, announcements) to passages ofparagraph length (e.g., excerpts from newspaper or magazine articles, personal and officialcorrespondence, popular stories). All test questions deal with either the factual content of thepassage or with straightforward inferences based on the passage. There are no texts or questionsrequiring a background in Polish literature.

The Structure section has two parts. In the first, examinees must choose from among fouroptions the correct word or phrase that is missing from a text. In the second, examinees mustindicate from among four underlined words which one contains an error.

For more information on the content of the PPT, see the section on Content Validity on page10 of this manual. The PPT Examinee Handbook, available from CAL, contains examples of thetypes of items found on the PPT.

Testing Time

The PPT is designed so that each section may be separately administered. The testing timefor the Listening Comprehension section is 40 minutes, with 50 minutes allotted for the ReadingComprehension section and 30 minutes for the Structure section. Including time for instructionsand breaks, the entire test may be administered in a single session lasting approximately two anda half hours, or in two sessions of approximately one hour and fifteen minutes each. Programsplacing a high emphasis on developing proficiency and/or decreased importance on grammarmay decide to omit the Structure section of the PPT. If it is omitted, the required testadministration time is decreased by about 30 minutes.

Complete information on administering the PPT is contained in the PPT Supervisor'sManual, which is sent with test orders. If desired, it may be separately requested from CAL.

USES OF THE PPT

The primary focus of the PPT is on assessing the examinee's ability to understand spokenand written Polish encountered in contemporary, real-life language-use contexts. As such, amajor use of the PPT is to provide an objective measure of an individual's general proficiencyin Polish. Another major use of the PPT is to provide Polish-teaching institutions with anexternal measure of the overall effectiveness of their instructional programs. A number of otheruses are possible as well. The PPT may be used for the following purposes:

admission to a Polish study programplacement within a Polish study programexemption from a Polish language requirementapplication for a scholarship or an appointmentcompetency testing upon exit from a Polish programcertification of Polish language proficiency for career placementevaluation of a Polish instructional program

2

CAL encourages use of the test by an institution if such use will assist it in making validdecisions in one or more of the above applications. However, the institution itself mustdetermine whether the PPT is an appropriate test with respect to its intended uses and mustestablish its own level of acceptable performance on the test.

PPT score users are invited to consult with CAL testing staff about current or intended usesof test scores. Furthermore, the testing office can assist institutions contemplating use of the testby providing information about the applicability of the test to particular situations and byproviding assistance in establishing performance criteria on a local basis. If it is necessary tocontact CAL, correspondence should be directed to:

Coordinator, Polish Proficiency TestCenter for Applied Linguistics1118 22nd Street, NWWashington, DC 20037TEL: (202) 429-9292FAX: (202) 659-5641

INITIATING TEST ADMINISTRATION

Registering for the Test

The PPT is available to colleges, universities and other institutions in the United States andabroad for administration to their students on tests dates of their choice. Administration may beinitiated in two ways. An institution may decide to administer the test to its students. In thiscase, the appropriate person makes arrangements with CAL directly. Alternatively, an individualwho needs to be tested may initiate administration. In this case, the individual must first contactCAL for the name of the nearest PPT testing center. If a test center is not available nearby, theindividual should contact a local institution that offers a Polish language program. Thatinstitution may then apply to CAL to administer the PPT to the individual by designating asupervisor and filling out an order form (see Ordering the Test below).

When registering for the PPT, an examinee is required to present a photo identificationdocument (e.g. driver's license, student I.D. or passport) to the supervisor. The supervisor mustverify the identification and record the name and social security number. The $20 testing fee ispayable either at the time of registration or on the test date. No examinee should be allowed totake the PPT without paying the fee in advance. The testing fee can be paid in cash or by acheck payable to the test center supervisor or to the institution. An invoice for the total testingfee will be issued to the institution after the test has been administered and test materialsreturned to CAL.

Ordering the Test

To place an order for the PPT, the supervisor must provide CAL with the followinginformation in writing:

1. title/position and affiliation of the supervisor;2. approximate test date;3. number of examinees expected;4. whether rush delivery is desired (at the institution's expense).

A test order should arrive at CAL approximately 2-3 weeks before the intended test date toallow timely shipment of the necessary test materials. To expedite processing of test orders, itis highly recommended that printed PPT Order Forms be used. A copy is printed on the lastpage of this manual. Telephone orders are accepted only under urgent circumstances and onlyfrom institutions which have administered the PPT previously.

Upon receipt of the Order Form, CAL will forward to the designated supervisor therequested number of PPT test booklets and associated materials, including the test tape for theListening Comprehension section, examinee answer sheets, PPT Examinee Handbooks, the PPTSupervisor's Manual and other materials. The PPT Supervisor's Manual contains all instructionsfor the actual administration.

It is the responsibility of the supervisor to safeguard and maintain the security of the testmaterials from the time they are received at the institution until they are returned to CAL. It isimportant that the test be administered in strict accordance with the procedures outlined in thesupervisor's manual. All materials requested on the Test Materials Checklist must be accountedfor and returned to CAL (see the PPT Supervisor's Manual for detailed instructions).

Score Reporting and Billing

Approximately 2-3 weeks following receipt of the returned test materials, CAL will forwarda roster of scores to the institution. An invoice for the total testing fee, including any deliveryexpense for rush orders, will follow shortly afterward. This invoice must be paid by check ormoney order, payable to: Polish Proficiency Test/Center for Applied Linguistics.

4

96

INTERPRETATION OF TEST SCORES

The Meaning of Scores on the PPT Scale

Rather than receiving a raw score on each section of the test (i.e., the number of rightanswers), examinees taking the PPT receive scores reported on the PPT Scale. A score on theNis Scale more accurately reflects an examinee's proficiency than the number of right answerson each section of the test. In addition, the scaled score contains interpretative information,which is explained below. The PPT Scale was developed using Item Response Theory (IRT),using the Rasch model. Scores on the PPT Scale are based on the proficiency demonstrated bythe examinee in terms of the difficulty of the items the examinee answers correctly. No penaltyis imposed for guessing, or for incorrect or blank responses.

Scores on the PVT Scale are interpreted in a normative manner. A score of 100 on anysection reflects the average (mean) performance of the PPT norming group. Each standarddeviation in this norming group is equal to 20 points on the PPT Scale. An examinee with ascore of 100 on any section is of average proficiency based on the norming sample describedbelow. An examinee with a score of 140 is two standard deviations above the mean, while anexaminee with a score of 80 is one standard deviation below the mean.

Description of the Norming Group

The PPT norming group is a sample of 70 examinees typical of North American studentsstudying Polish at the college level in any given year. At the time of forming, 67% percent ofthe group were in their first year of Polish study, 24% were in their second, and 9% were intheir third (or higher) year. Forty-eight percent were male and 52% were female. Fifty-eightpercent were of Polish ethnicity. Sixty-four percent of the group indicated that they spoke noPolish at home, while 3% spoke Polish at home "most of the time," 6% spoke Polish at home"less than half the time," and 27% spoke Polish "very infrequently" at home.

Quartile Distributions

Because of the small number of Polish examinees who could be involved in the norming ofthe PPT, it is impossible to give accurate norms at every score level. Table 1 provides quartilescores on the PIY1' Scale for examinees in the first, second, or third or more year of Polishlanguage study. The figures are based on the results of all 129 examinees kv'ao participated inthe field testing of the PPT. However, not every examinee took every section of the PPT. Thecharacteristics of the entire group of 129 examinees did not deviate substantially from those ofthe 70 examinees who comprised the norm group, described above. The norm group is a subsetof the entire group of 129 examinees.

Quartiles represent the bottom, second, third and top quarter of the score distribution. Ascore in the second quartile indicates that the examinee performed in the second quarter of allstudents in his or her subgroup, and better than the bottom 25 percent of all the examinees in

5

9i

that subgroup. A score in the fourth quartile indicates that the examinee scored in the top 25%of all other examinees in the subgroup; i.e., better than 75% of all other examinees.

Table 1Quartile Distributions for the PPTby PPT Section and Year of Study

First Year Second Year Third Year or Higher

Quartile List Read Str List Read Str List Read j---1.

Sir

4th(76-100%)

99 andabove

105 andabove

106 andabove

121 andabove

119 andabove

123 andabove

139 andabove

144 andabove

154 andabove

3rd(51-75%)

92-98 93-104 93-105 105-120 107-118 110-122 130-138 127-143 122-153

2nd(26-50%)

83-91 83-92 84-92 97-104 99-106 104-109 101-129 109-126 108-121

1st(1-25%)

82 andbelow

82 andbelow

83 andbelow

96 andbelow

98 andbelow

103 andbelow

100 andbelow

108 andbelow

107 andbelow

Examinees (56) (60) (69) (18) (17) (19) (6) (15) (15)

To use Table 1 in interpreting an examinee's score, first identify the examinee as either afirst, second, or third (or higher) year student. Then, for each section, find the score group inthe table that contains the examinee's score. Then read the quartile group from the left handcolumn that corresponds to the examinee's score.

Example 1Susan is a student in a first year college-level Polish class who received a PPT Scalescore of 95 on the Listening Comprehension section. Under the column for first yearstudents, we find the column for listening, and under this column, we find the scoregroup that contains her score (92-98). This corresponds in the left hand column to thethird quartile. Thus, we know she has scored above 50% of all first-year examinees inthe Listening Comprehension section, but below the top 25% of students in first year

Example 2Michael is a student in a third year college-level Polish class who received a PPT Scalescore of 125 on the Reading Comorehension section. Under the column for third yearstudents, we find the column for reading, and under this column, we find the score groupthat contains his score (109-126). This corresponds in the left hand column to the secondquartile. Michael has scored above 25% of all second-year examinees in the ReadingComprehension section, but below the top 50% of third-year examinees. Note, however,that if Michael had been in the second-year category, his Reading Comprehension sectionscore of 12f would have placed him in the fourth quartile, indicating that he scored

6

96

above 75% of the second year students, and in the top 25% of all second-year examineesin reading.

Using the Group Means for Polish, non-Polish Speakers

Some examinees in the norm group indicated that Polish was used in the home more thanhalf the time. Such examines may have an advantage on a test of general language proficiency.However, since these examinees formed only a small minority of the total, and since they docomprise a part of the total group of North American students studying Polish, they were notexcluded from Table 1. Nonetheless, PPT users may want to understand the meaning of anexaminee's test scores in terms of whether the examinee indicates that much Polish is spokenat home or not. Table 2 presents the average PPT Scale score for each section by year of studyfor examinees who indicated that they do not generally speak Polish at home (labeled English)and those who indicated that Polish is spoken about half the time or more (labeled Polish).Underneath each average score is the number of examinees upon which the mean is based. Themore examinees, the more reliable the mean score. Means for subgroups with less than 3members are not shown.

Table 2Mean Scores on the PPT Scaleby Level and Home Language

(Number of Examinees in Parentheses)

Listening Reading Structure

Year of Study English Polish English Polish English Polish

First Year 92(53)

---0

93(55)

96(5)

94(64)

96(5)

Second Year 107(15)

---

(-)

107(15)

---(-)

110(16)

- --

(-)

Third Year(or higher)

124

(5)

__,

(-)119

(7)130(8)

121

(7)129(8)

To use Table 2, find the group to which the examinee belongs and compare his or her scoreto that group's mean score.

Example 3Susan, the first-year PoLsh student who received a score of 95 on the ListeningComprehension section of the PPT, speaks no Polish at home. Her score is slightly abovethe average score for her peers.

7

99

Example 4Michael, the third-year student who scored 125 on the Reading Comprehension section,indicated that he generally speaks Polish at home. His score places him slightly belowthe mean of 130 for his peers. On the other hand, if he belonged to the "English"category, his score of 125 would have been slightly above average.

Performance of Polish Speakers on the PPT

The PPT is intended for English- speaking learners of Polish. Since the questions and optionsfor the Listening and Reading sections as well as the directions for the entire test are in English,knowledge of English is required to take the test. Thus, it is impossible to use these tests toexamine the performance of native Polish speakers (or speakers of a language other thanEnglish) who do not have a good knowledge of English. Test results may not be accurate forany examinee who does not have a good command of English; i.e., a level generally equivalentto that of a first-year college student enrolled in the regular program of an American university.

In the field testing of the PPT, 17 of the 129 examinees indicated that they were of Polishethnicity and spoke Polish at home "most of the time." The responses of these examinees werecarefully examined to see to what extent, if any, these examinees were getting easier itemsincorrect; i.e., items that they may have missed because of their restricted ability in Englishrather than their proficiency in Polish. None of the responses of the Polish speakers showed anyincongruities on any of the three sections of the test. On the basis of experience to date, it isreasonable to assume that the level of English used in the test will not pose a problem to thenative P,31ish speaking examinee who has enough proficiency in English to be enrolled at anEnglish-speaking university in North America.

Availability of Institutional Interpretation of Test Scores

In addition to using the normative data provided in this manual, institutions may wish tocarry out further analyses of the scores on a local basis. CAL staff would be pleased to discusssuch analyses with interested institutions and to facilitate these activities as far as possible.

STATISTICAL CHARACTERISTICS OF THE TEST

Reliability

The reliability of a test is the extent to which it yields consistent results. Thus, high testreliability is desirable.

Table 3 gives the Rasch equivalent of the KR-20 reliability of each section of the final formof the PPT based on the data from examinees involved from the field testing. The number ofexaminees for each section is: Listening (96), Reading (110), and Structure (123).

8

100

Table 3Reliability of thePPT by Section

Section ReliabilityListening Comprehension .92

Reading Comprehension .90

Structure .88

The figures presented in Table 3 indicate that the PPT is a highly reliable tests which canbe used with confidence by programs needing trustworthy measures of Polish languageproficiency.

Precision of Measurement

Any measurement involves a degree of error. The higher the precision of the measurement,

the smaller the error. In Classical Test Theory, the degree of error is usually indicated by thestandard error of measurement (SEM). One of the limitations of this approach is that itrepresents the SEM as being the same for all examinees. Test scores, however, are unequallyprecise for examinees at different levels of ability. Item Response Theory (IRT), used in scoring

the PPT, also provides a measure of the precision of each score on the PPT Scale. In IRT, thismeasure is called the standard error of the ability estimate. Appendix A contains the standarderrors for PPT scores, rounded to the nearest whole number.

A traditional SEM may also be calculated for the PPT on the PPT Scale. These are presentedin Table 4. Figures have been rounded to the nearest whole number.

Table 4Standard Error of Measurement (SEM)

for the PPT(in Rounded Scale Score Units)

Section SEMListening 6Reading 7Structure 7

Either the SEM from Table 4 or the standard error of the estimate from Appendix A givean indication of how accurate a single test score is. Either can be used to construct confidenceintervals around PPT scaled scores. The confidence interval takes measurement error into

account. One way to view this confidence interval is to say that the examinee's true score (i.e.,the average score an examinee would receive if it were possible to give him or her the test avery large number of times) would lie, with 67% certainty, within plus one or minus one

9

101

standard error of the given score. Another way to view this is to say that on a re-test, if therehas been no change in the examinee's true proficiency, the examinee will score within plus orminus one standard error of his or her score about 67% of the time.

Example SSusan received a score of 95 on the PPT Listening section. Using Table 4, we can saythere is a 67% probability that her true score lies between 89 and 101 (95 minus 6 and95 plus 6). Using the more precise estimate from Appendix A, we would say there is a67% chance that her true score is between 90 and 100 (95 minus 5 and 95 plus 5).

A careful examination of Appendix A reveals that the most precise area of measurement onthe PPT lies between 66 and 136. For scores above 136, the measurement precision of the PPTrapidly diminishes. A score of 136 indicates a performance well above the cut-off point for thefourth quartile for first and second year students, and in the third quartile for students in thecategory third year and above. In other words, the measurement precision of the PPT should notbe of concern in interpreting the scores of first and second year Polish students on any sectionof the test. However, test score users should be aware that the PPT may not be able todiscriminate accurately between very high scoring examinees on the PPT; i.e., examinees whoscore above 135 on any section. Table 2 indicates that such examinees would generally beindividuals who have completed three or more years of Polish study and who speak Polish athome.

Validity

Validity refers to the extent to which a test actually measures what it purports to measure.The PPT claims to measure an examinee's proficiency in understanding authentic spoken andw.:tten Polish, and ability to deal with Polish structure.

The validity of any test cannot be "proven;" it can only be established by the collection ofevidence that the test is indeed measuring what it purports to measure. Some commonly acceptedtypes of evidence include evidence for content, concurrent, and construct validity. Each type ofvalidity, and evidence which supports it in the case of the and the PPT, are explained below.

Content Validity

Content validity is based on test content and refers to the extent to which the items in a testadequately sample the linguistic and behavioral domains of the construct that the test is meantto measure. The PPT is intended t3 be a measure of proficiency in dealing with every-day "real-life" Polish. Validity based on content thus entails an examination of the degree to which thetests, in their stimulus passages, sample from the corresponding language-use situations theexaminee might be expected to encounter in real life.

10

102:

Content Domains

The ACTFL Guidelines for listening and reading postulate certain content and functionaldomains as appropriate to the proficiency of an English-speaking learner of Polish at differentlevels of proficiency. The developers of the PPT used authentic listening and reading passagesand wrote corresponding items to reflect these domains. Content and functional domains can begrouped into three categories, which represent increasing levels of proficiency in Polish as asecond language. These levels may be designated as:

Intermediate LevelSurvival. At this level, the learner can *survive" by performing simpledaily tasks such as shopping for food, clothes, and incidental necessities. PPT listeningcomprehension items at this level require the understanding of oral language typically heardin locations such as a grocery store, train station, bank, bus, and airport. PPT readingcomprehension items at this level include certain signs, advertisements, storefrontdesignations, simple messages -h a social purpose, and short straightforward instructionsencountered in daily life.

Advanced LevelTourist Resident in Polish Society. At this level, the learner canunderstand social conversation on everyday topics and can read short, descriptive narrations.PPT listening comprehension items at this level require comprehension of short conversationsbetween two people dealing with general, every day topics, or the comprehension of passagessuch as news items and reports dealing primarily with factual information. PPT readingcomprehension items at this level require the examinee to comprehend main ideas and somefacts from passages excerpted from newspaper articles, public notices, personalcorrespondence, and popular stories.

Superior LevelForeign Professional in Polish Society. At this level, the learner cancomprehend oral and written language on virtually any matter of general interest to educatednative speakers in authentic Polish situations. PPT listening stimuli at this level involveconversations between two speakers dealing with topics of a general academic nature suchas recent advances in health care and current events, or longer discourse excerpted fromspeeches or reports. Questions may deal with the speaker's opinion or attitude, or they mayrequire inference on the part of the examinee. Reading stimuli at this level are generallylonger and more technical and are followed by a questions dealing with basic factualinformation covered in the passage, or inference or analogy based on information presentedin the passage. The passages cover a wide variety of daily topics such as those that may befound in a newspaper or magazine, e.g., results of a public opinion survey, job descriptions,and vacation travel.

For the Structure section, the PPT tests knowledge of correct Polish syntax. Unlike listeningor reading comprehension, the Structure section does not involve real-life language-use tasks perse, though clearly knowledge of Polish syntax is a part of comprehending spoken and writtenPolish. Items testing recognition of Polish syntax on the PPT are of two types. The first typetests knowledge of correct structure by requiring the examinee to choose from among fouroptions the correct word or phrase that should be inserted in a blank within a printed paragraph.

The second type tests structural knowledge by requiring the examinee to find from among fourunderlined words the one that is grammatically incorrect. In this section of the PPT, only Polishis used for both the stimulus and response options.

Appendix B contains a list of the comprehension tasks and topic areas covered in the PPTlistening and reading comprehension sections. It also contains a list of speaking functionscovered on the listening section of the test. Appendix C gives information on the grammar topicscovered in the structure section of the PPT.

Authentic Sources

Stimulus passages for listening, reading and structure on the PPT were drawn from authenticlanguage sources. Item developers collected passages for the reading comprehension section fromsources such as Polish language newspapers, magazines, journals, books, street signs, schedules,and tickets. For stimulus passages for listening comprehension, they listened to anci transcribedlistening passages from sources such as Polish language news broadcasts, movies, andannouncements. These texts were modified only to the extent that they needed to be clarifiedwhen taken out of the larger context. Questions were designed to check comprehension of themeaning of passages, and, for more difficult passages on the PPT, to check comprehension ofopinions, attitudes or inferences contained in the passages. A representative list of sources forthe listening and reading passages used on the PPT is contained in Appendix D.

In terms of content validity, then, it can be demonstrated that the PPT items have beendrawn from real-life use of Polish language.

Concurrent Validity

A second type of evidence of validity is concurrent validity. Concurrent validity refers to theextent to which a test score correlates with results that may be obtained through the use ofindependent criteria external to the test, measured at the same point in time, to see if expectedrelationships exist.

For the PPT, the reported level of Polish study may be use as an external criteria may beused. It would be expected that scores on the tests would increase as amount of study increases.Both Tables 1 and 2 indicate that this is the case for the PPT. In each skill, the median (Table1) and average (Table 2) score increases as the level of Polish instruction increases. These tablesthus provide evidence of the concurrent validity of the PPT as a measure of Polish languageproficiency.

Construct Validity

A third way of examining validity is construct validity. The goal of construct validity is todetermine whether or not a test measures a single underlying trait. One assumption of the IRTmodel used to develop and score the PPT is that the items are munidimensional"; that is, onlyone examinee ability or trait is necessary to account for performance on the test. The fit statistics

12

104

provided by the IRT analysis provide evidence of the extent to which unidimensionality exists.Although perfect fit is impossible, it may be argued that if the overwhelming majority of itemsare appropriately fitting, then there is strong evidence for the construct validity of the test. Foreach :t4tm, a standardized outfit statistic is calculated. One common criterion is to consider anyitem for which this statistic is above 2.00 or below -2.00 as misfitting. Using this criterion, only3 of the 50 listening items (6%) and 2 of the 50 reading items (4%) on the final form weremisfitting. This high degree of fit provides strong evidence for the construct validity of thesemeasures. In the structure section, 5 of the 35 items (14%) were misfitting. Although 14%represents a higher proportion of misfitting items than for the listening and reading sections, thefigure is low enough to lend some support the construct validity for the structure section. It maybe noted that one of these five items has been completely revised in the final form.

Validity Swnntary

The above discussion provides evidence for the validity of the PPT. CAL plans to continueresearch pertinent to the validity of the test and will update this section of the manual asappropriate.

Intercorrelations Among Test Subscores

The three sections of the PPT are designed to measure different skills within the generaldomain of Polish proficiency. It is expected that these skills are interrelated; i.e., persons whoare highly proficient in one skill area will tend to be proficient in the other skills as well.However, the intercorrelations are not expected to be perfect. If they were, there would be noneed to report scores for each section; the subscores would represent the same rather thandifferent aspects of language proficiency.

Table 5 shows the Pearson product-moment correlation coefficients measuring the extent ofrelationships among the three subsections of the PPT based on scores on the final form of thePPT for the 86 examinees who took all three sections in the field testing.

Table 5Intercorrelations Among Subscores

Lig Read 511Listening -Reading .84Structure .82 .92

Table 5 indicates that there is a fairly strong relationship among the skills tested by the threesubsections of the test. This relationship is strongest between reading and structure, which maybe expected in a highly inflected language as Polish. Nevertheless, it is reasonable to say thateach subsection scores provides some unique information about the examinee's proficiency in

13

105

the Polish language. This pattern of intercorrelations between the test scores helps support thevalidity of the constructs the PPT claims to measure.

For Further Information

As further data becomes available, CAL intends to update the statistics presented in thismanual and supply with additional information pertinent to users of the PPT. In the interim, anyquestions or comments about the PPT may be directed to: Coordinator, Polish Proficiency Test,Center for Applied Linguistics, 1118 22nd Street, NW, Washington, DC, 20037.

REFERENCES

American Council on the Teaching of Foreign Languages. (1986). AC7fL proficiencyguidelines. Hastings-on-Hudson, NY: Author.

14

106

Appestat AStandard Error of Estimation

Polish Proficiency Test

Scaled Score Listening Reading Structure71 25 26 27

170 24 25 27169 23 25 26168 22 24 25167 22 23 24166 21 22 23165 20 22 23164 20 21 22163 19 20 21162 18 20 21161 18 19 20160 17 19 19159 17 18 19158 16 18 18157 16 17 18156 15 16 17155 15 16 17154 14 16 16153 14 15 16152 13 15 15151 13 14 15150 12 14 14149 12 13 14148 12 13 13147 11 13 13146 11 12 13145 11 12 12144 10 12 12 I

143 10 11 12142 10 11 11141 10 11 11140 9 10 11139 9 10 10138 9 10 10137 9 10 10136 8 9 9135 8 9 9134 8 9 9133 8 9 9132

IMIN8 8 9

131 7 8 9I

130 7 8 8129 7 8 8

1

128.... 7 8 8

127 7 8 8126 7 7 8125 7 7 7124 6 7 7123 6 7 7122 6 7 7121 6 7 7120 6 7 7119 6 6 7118 6 6 7117 6 6 6

107

CP

.

Os .4

00'0

.4 0-

-4 -

.100

0000

0000

00 0000 so

vs00

80

a-4

000

.16

a a

Vs

Vs

VIVi

Vs

VIVi

Vi

VI

Vs

Vi

Vi

Vs

Vi

Vs

Vs

Vs

Vi

Vi

Vi

Vs

Vs

Vi

Vi

Vs

Vi

VIVi

Vi

Vs

Vs

Vs

Vs

Vi

VsVi

Vt

Vs

Vf

Vt

Vs

VsVi

Vi

Vs

Vt

Vt

Vi

Us

Vs

Vs

Vi

Vi

Vi

Vi

Vs

Vi

Vi

VI

Vi

Vs

Us

VtVI

VI

Vs

Vs

VsVi

Vi

Vi

Vi

Vs

Vs

Vi

Vi

Vi

Vs

Vs

Vi

Vs

Ve

ON

Oi

Os

Os

Oi

i00

00 -

4 1

-4 -

4 -4

-J

-.1

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Os

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Vs

Os

Os

Os

Os

0% O

s O

s O

s O

s O

ss O

s O

sO

s

R

Appendix B

Comprehension Tasks for the PPT Items

Each item in the listening and reading comprehension section was classified by the itemdevelopers as to the type of comprehension task required to answer the item correctly. Threebroad areas of comprehension were used: understanding the main idea, understanding a fact ordetail, or understanding an inference.

Items identified as requiring the examinee to understand the main idea typically require thegeneral comprehension of the entire passage. In same cases, the main gist of the passage maybe grasped with the comprehension of one or two key words or phrases. In the majority of theitems, however, comprehension of the main idea comes only if most of the passage is correctlyunderstood.

Items identified as requiring the examinee to understand a fact or supporting detail generallyrequire comprehension of the main idea and some supporting detail. However, in some non-prose items from authentic materials (e.g., advertisements), an item in this category may entailthe comprehension of isolated words.

Items identified as requiring the examinee to understand an inference require a thoroughcomprehension of the entire passage in order to identify a correct implication based on thepassage but which is not stated directly in it.

Comprehension Task Listening Section Reading Section

Understand the Main Idea 23 items,

20 items

Understand a Fact orSupporting Detail

20 items 26 items

Understand an Inference 7 items 4 items

103

Topic Areas Covered on the PPT

Each passage used in the listening and reading comprehension section was classified by thetest developers in terms of the topic area which best described its content and vocabularydomain. Since the PPT is a test of general proficiency in Polish, these topic areas are to beunderstood broadly. In other words, no examinee would need particular expertise in any ofthese topic areas to answer correctly an item related to that topic area. On the other hand, thistable gives an indication of the distribution of the general vocabulary and content found on thePPT in terms of broad categories. Although there is often much overlapping between categories,each item has been placed in only I category in this table.

Topic Area Listening Section Reading Section

Art 1

Business 1

Consumerism 7

Dates 1

Economics/the Economy 3

Education 3 6

Entertainment 5 5

Everyday Life 3 1

Family 3

Food

Geography

Government 1 1

Health 5 3

History 1

Housing 1

Information Listing 1

Politics

Social Exchange

Social Life

Society 1 1

Sports 4 2

Transportation 3

Weather

Other

110

Speaking Runctions found in the PPT Listening Comprehension Passages

Each passage in the listening comprehension section was classified by the test developers interms of the primary speaking function of the passage. In many cases, there are severalspeaking functions. Each item, however, was assigned to only ()Ie. Generally, it is the onemost connected with getting the item for the passage correct. Thus this table is meant to givean indication of the distribution of the speaking functions found in the PPT listeningcomprehension section.

Speaking Function Number of Items

Complaining 6

Describing 1

Expressing an opinion 2

Giving advice 1

Giving information 19

Making an inquiry 11

Interviewing 1

Making a comment 1

Making a request 2

Making a purchase 1

Making an introduction 1

Making an invitation 1

Negotiating 1

Persuading 1

Showing preferences 1

1 1 i

Appendix C

Grammatical Areas Covered in the PPT Structure Items

Each item in the structure section of the PPT was classified as involving primarily either onepart of speech or another aspect of structure (such as negation). This table shows the numberof items involving those primary forms. Specific aspects of each form (such as case, numberagreement, gender, tense) are not listed separately.

Form Number of Items

Nouns 11

Verbs 7

Adjective/Adverbs

Pronouns 5

Prepositions 3

Other 3

Appendix D

Sources for PPT Passages

This table lists the most widely used types of sources for passages found on the PPT.

Listening Passages Reading Passages

AdvertisementsCommentariesInterviewsMoviesRadio News ReportsRadio AnnouncementsRecorded ConversationsTelevision News ReportsTelevision AnnouncementsTelevision ProgrammingWeather Forecasts

113

AdvertisementsBulletinsEditorialsInstructionsLabelsLettersMagazine ArticlesNewspaper ArticlesPopular NovelsSchedulesSigns

POLISH PROFICIENCY TESTOrder Form

Ship to: (Please print) Bill to: (if different from left)

Name

Position/Title

Department

Institution

Street

City/State/Zip Code

Telephone Number

Name

Position/Title

Department

Institution

Street

City/State/Zip Code

Telephone Number

First order?Yes (If yes, CAL reserves the right to verify the above information.)No

Number of examinees expected: ($20 per examinee)

Purpose and me of the test: (Check all that apply)admission to an Polish study programplacement within an Polish study programexemption from an Polish language requirementapplication for scholarship or appointmentcompetency testing upon exit from an Polish programcertification of Polish language proficiency for career placementevaluation of an Polish instructional programother (Please specify:

Approximate testing date: Required delivery date:

Payment:You will be billed for the actual number of examinees tested after you return the test materials to CAL.

Delivery charges will be paid by CAL. provided this Order Form reaches CAL at least two weeks prior to therequired delivery date indicated above. Delivery charges for rush orders requiring a shorter delivery time will bebilled to the institution placing the order. Official score reports will be issued about two weeks after the materialshave been returned to CAL.

Security AgreementIt is vital that you maintain the security of the test materials from the time you receive them until they are

returned to CAL. You must agree to: (1) keep the PPT test booklet(s) and test tape(s) confidential and secure, and(2) guarantee that none of the test booklet(s) or test tape(s) will be reproduced in any form. Please signify youracceptance of these terms by signing your full name below.

Signature Date

Date received:

Office Use Only

Date filled: Processed by:

114 BEST COPY AVAILABLE

Appendix F

Forms for the Operational Testing Program

POLISH PROFICIENCY TESTOrder Form

Ship to: (Please print) Bill to: (if different from left)

NameName

Position Title

Department

Position Title

Department

Institution Institution

Street Street

City / State/Zip Code City/State/Zip Code

Telephone # Telephone #

First order?Yes (If yes, CAL reserves the right to verify the above information)No

Number of examinees expected: ($20 per examinee)

Purpose and use of the test: (Check all that apply)admission to a Polish study programplacement within a Polish study programexemption from Polish language requirementapplication for scholarship or appointmentcompetency trsting upon exit from a Polish programcertification of Polish language proficiency for career purposesother (Please specify:

Approximate testing date: Required delivery date:

PaymentYou will be billed for the actual number of students tested after you return the test materials to CAL. Delivery

charge will he paid by CAL, provided this order form reaches CAL at least two weeks pnor to the required deli%erydate indicated above Delivery charges for rush orders requiring a shorter delivery time will be billed to theinstitution placing the order. Official score reports will be issued about two weeks after the materials have beenreturned to CAL

Security AgreementIt is sital that you maintain the secunty of the test matenals from the time you receive tbem until they are

returned to. CAL You must agree to: (1) keep the PPT test booklet(s) and test tape(s) confidential and secure, and(2) guarantee that none of the test booklet(s) or test tape(s) will he reproduced in any form. Please signify youraci.eptan.e of these terms by signing your full name below.

Signature

Date receivedOffice Use Only

Date filled Processed by

116 BEST COPY AVAILABLE

1Date

1

LOL SH PROFICIENCY TEST

Test Materials(Test Order Kit)

Supervisor: Date:

Institution: Test Center Code:

LECEELKLIESIAMMEital:

The following materials should be contained in this package. Please check all materials receivedagainst this list. If there is any discrepancy, please contact the Polish Proficiency Test Office(202/429-9292) immediately. An marks those items which must be returned.

EnclosedDescription Quantity in return

envelope?'1. Test Materials Checklist

2. Order Form

*3. Supervisor's Manual

4. Examinee Handbook

'5. Polish Proficiency Test Booklet(Serial numbers to

"6. Polish Proficiency Test Tape(Serial numbers

'7. Standard Answer Sheet

es. Supervisor's Report Form

9. Test Interpretation Manual

10. Pre-addressed First-Class Return Envelope

IMPORTANT: The Supervisor's Manual must be carefully reviewed prior to the administrationdate. The Examinee Handbook(s) should be distributed to students well ahead of the testingdate.

RETURNING TEST MATERIALS:Within three days after the test has been administered, the materials marked with an asteriskabove must be returned to the Center for Applied Linguistics. For each item returned, pleasemark a check on this form and enclose it in the return envelope. Make sure the quantity of eachitem matches what was sent in the original shipment. Thank you.

Office Use Only

Date sent: Date returned: Processed by:

117

POLISH PROFICIENCY TEST

Supervisor's Report Form

This report form is used for reporting any irregularities in test administration and for providingany comments or suggestions you may have for improving the Polish Proficiency Test. Pleasefill out this form, sign your name, and return it with the test materials. Thank you.

* * * * * * * * *

Test Security

*

By agreeing to serve as the test supervisor, I am responsible for ensuring the security ofthe test. I have kept the test materials confidential and secure at all times. None of the testbooklets or test tapes have been reproduced in any form.

Irregularities:

Test Administration

The test was administered in exact accordance with the procedures described in theSupers isor's Manual. Any deviations from the stated procedures are listed below.

Irregulanties:

Conditions of Test Materials

Before returning the test materials, I have checked the condition of the test booklets andtest tapes. All materials are being returned in their original condition. All materials listed onthe Test Materials Checklist have been enclosed.

Irregularities:

Comments'Suggestions

(In order to continue to meet your testing needs, it is ver) important that youcommunicate with us any comments or suggestions you may have regarding the PolishProficienc) Test. Please use the back of this form or attach a separate sheet if necessary.)

Name Institution

Signature Date

Appendix G

Minutes of First Local Development Committee Meeting

MINUTES

LOCAL TEST DEVELOPMENT COMMITTEEPOLISH PROFICIENCY TEST PROJECT

OCTOBER 18 aad 19, 1991CENTER "OR APPLIED LINGUISTICS

WASHINGTON, DC

Committee Members and CAL Staff Present:k,.

t,

Dobrochna E. Dyrcz-Freeman . . . (Formerly) Princeton UniversityRafal M. Kwiatkowski (Formerly) Georgetown UniversityGrazyna Z. Privorotsky CAL, George Washington UniversityKrystyna M. Sadlowska Foreign Service InstituteWaldemar E. Walczynski

,CAL

Xixiang Jiang * CAL;Dorry Mann Kenyon CAL

Charles W. Stansfield 4 CALt,: 1-

Introduction and BackgroundThe members of the Committee introduced themselves. TheCommittee is comprised of Polish language experts, who areexperienced instructors and researchers. This group is assistedby a CAL Supervisory group. Charles Stansfield serves as ProjectDirector; Dorry Kenyon serves as Assistant Project Director, andXixiang Jiang is Project Coordinator.

Stansfield, in his introductory remarks, noted the increasedinterest in Polish language study and explained the need for anational standardized proficiency test. It is hoped that thistest, once developed, will serve as a catalyst for theimprovement of Polish language instruction and learning.Stansfield also described CAL's Chinese Proficiency Test (CPT),which has been used by colleges and high schools across thenation. This proficiency test, along with its second form calledPre-CPT, has been well received in the Chinese language teachingprofession both for its validity and applicability. The CPT willserve as a model for the development of the Polish ProficiencyTest.

project ScheduleKenyon presented the time frame for the test development project.He no '-ed the importance for the draft test form to be ready intime for field testing in January/February of 1992, and the finalversion to be normed before school breaks for vacation (May,1992). The first draft test items are expected to be ready bythe middle of December. From then on, there will be various peerreviews and revisions. The External Advisory Committee memberwill complete their review of the items in January 1992. Fieldtesting is planned for February. Members of the ExternalAdvisory Committee will be called upon to assist in arranging forfield testing at their own institutions, if possible.Statistical analysis and revisions after field testing will be

126

1

done in March and April. The norming form of the test will beadministered in between April and May. In the months to follow,the final form of the Polish Proficiency Test will be assembledand operationalized. The Project Final Report will be completedby the end of September 1992.

Target LevelsIn the discussion of the target levels for the Polish languagetest under development, several issues were addressed. The firstissue was which proficiency level to choose as the target levelfor the test. In view of the difficulty of the language itselfand the limited nature of the exiiting teaching materials, it wasagreed that the appropriate target levels for this test shouldrange from Novice-High through Advanced-Plus on the proficiencyscale of the American Council for the Teaching of ForeignLanguages (ACTFL). This range is equivalent to 0+ through 2+ onthe proficiency scale of the Federal Interagency LanguageRoundtable (FILR). The second issue was which student body tochoose as the target group for this test. Noting that mostPolish language teaching programs do not continue beyond thesecond year and that students in area studies at the graduatelevel usually enter beginning Polish programs at theundergraduate level, it was agreed that the envisioned targetstudent group for this language test includes Polish languagestudents in their first or second year of college and governmentor business professionals the intensive language trainingprograms.

Test DomainsThe difference between an achievement test and a proficiency testwas reviewed. It was recognized that there are several Polishlanguage textbooks in wide use and that, due to the nature of thelanguage itself, there is a strong emphasis on grammaticalstructures in the teaching profession. It was also recognizedthat since the test under development is to serve as a nationalstandardized proficiency test, it should not be based on aparticular set of textbooks or language programs, though effortsshould be made to examine some of the books and programs duringthe test development process. Acknowledging that the knowledgeof syntactic structures is an integral part of communicativeability in the Polish language, it was determined that this testwill include a structure section, probably 20-25% (30 to 40items) of the entire test. There will be three sections in thefinished test: Listening Comprehension, Reading Comprehension andStructure.

Test Format and Item NumbersIn conformity with the current trend in large scale standardizedtests, the Polish language test will employ the multiple choicerather than open-ended answer format. Several item types werereviewed, including error recognition, sentence completion andlexical insertion. It was generally agreed that a four-option

12;

choice test type be used throughout the test. In both theListening and Reading Comprehension Sections, four answerchoices will be provided for each test question. Two types ofitems were discussed as possibilities for the Structure Section.The first is completion, in which the examinee would need tochoose the correct words that complete missing portions within aPolish text. The second item format was error recognition, inwhich the examinee must find the error in a line of Polish text(within a larger passage). Four words in a line would beunderlined, one of which contains a grammatical error. For thewhole test, there will be between 125 and 150 items, with roughlyequal numbers of List ling and Reading Comprehension items.

Having laid down the above guidelines and specifications, theCommittee proceeded to deliberate on issues pertaining to otheraspects of the project. The following concerns were addressed.

External Advisory CommitteeConsidering the small number of people currently listed in theExternal Advisory Committee, members of the Local TestDevelopment Committee felt that there is a need to increase thenumber and include Polish language professionals from a widerrange of institutions. Names were suggested and noted. AnExternal Advisory Committee with representatives from variouseducational and research backgrounds will enhance the testdevelopment efforts and render more professional assistance inthe process of reviewing test items and locating Polish languageprograms across the country.

Sources of Polish language StudentsIn order to validate and norm the test after initial development,there needs to be enough students to participate in the fieldtest and norming administration. The minimum preliminary numberof students for the field test is 200. Members of the committeeoffered names of people who could provide suggestions,locationsof various existing Polish language programs and institutionswith potential sources of students. After recruiting efforts areundertaken, a mailing list will be established, not only for usein the test development process, but also for disseminatinginformation dissemination on the Polish Proficiency Test once itis in its operational form.

Sources of Authentic MaterialsIt was quickly realized that materials for the ReadingComprehension are relatively easy to collect, since some membersof the committee have been accumulating Polish newspapers andother authentic materials for other projects. It was noted thatin addition to newspapers, item writers will collect otherauthentic materials in order to include language forms. Thesematerials will be drawn from novels, stories, magazines, personaland public documents. Efforts will be made to collect realia,including objects or signs often encountered in a real lifePolish situation. Gathering materials for the ListeningComprehension Section appeared to be not as easy, though some

122BEST COPY AVAILABLE

sources were quickly identified. Government agencies, theembassy, radio and TV companies were mentioned for theircollection of original audio tapes, videos of movies and tapes ofbroadcast programs. Items will be written using these authenticmaterials. However, the materials will be re-recorded by Polishprofessionals to avoid possible background noise and distraction.As was noted, for both reading and listening items, there will besome efforts required to collect authentic materials at the lowproficiency levels specified earlier.

Item Writing ProceduresOnce developed and approved, the test development guidelines andspecifications will serve as the principal framework from whichindividual item writers will work. It was agreed that the itemwriters will mainly work at their own rate to collect sourcematerials and write the assigned items. To ensure adequatecommunication and peer review, however, it was proposed that itemwriters submit their test items in keeping with the work schedulefor review and approval. CAL staff will confer with the itemwriters on a regular basis in critiquing and evaluating thesubmitted items. At the development stage, work will be dividedamong the item writers. Each individual will be responsible forcollecting authentic language materials. After theappropriateness of the selected materials has been determined bya process of peer review, each person will write the assignedtest items. For the field test version, the number of test itemswill be approximately 180 in all. Item writers are expected tokeep a record of the sources from which their items are composed.

After deliberations on these issues, the Committee started anitem writing training workshop. The item writers, with theassistance of Stansfield and Kenyon, reviewed some principles oftest item writing, evaluated some sample test items written in aPolish textbook, and finally practiced writing some readingcomprehension items. The peer review and critiquing process thatfollowed proved to be informative and beneficial for the itemwriters.

The meeting adjourned at 3:00 p.m. Saturday, October 19, 1991.No following meeting was scheduled, but members of the Committeeexpressed their availability for such an occasion, if needed.

Respectfully submitted,

Xixiang Jiang, Project Coordinator

Dorry Mann Kenyon, Assistant Project Director

Charles W. Stansfield, Project Director

123

Appendix H

PPT Item Submission Forms

124

READING COMPREHENSION ITEM SUSKISSION FORM Number:

Source

Text FormTopic AreaContest

Cognitive Task: To Understand( ) Learned Material ( ) Main Idea ( ) Facts/Detail( ) Implication ( ) Other:

Contextual Support ( ) Strong ( ) Moderate ( ) WeakVocabulary ( ) Easy ( ) Medium ( ) DifficultGrammar ( ) Easy ( ) Medium ( ) Difficult

Rey to Understanding ( ) Vocabulary ( ) Grammar( ) Context ( ) Pragmatics( ) Other:

Level of Difficulty RI, NM NH IL IM IH A A+ S============================ ====W==========WW===Stimulus (Polish):

(Translation)

Question or Stem:

Options: (* * Key)(A)

(B)

(C)

(0)

Author: Date:

123

LISTENING COMPREHENSION ITEM SUBMISSION TORN Number:

Source

Speaking Function(s)Topic AreaContext

Exchange Pattern ( ) U ( ) A-B ( ) R-B-A ( ) A-B-A-B ( ) M

Cognitive Task To Understand( ) Learned Material ( ) Main Idea ( ) Facts/Detail( ) Implication ( ) Other:

Contextual Support ( ) Strong ( ) Moderate ( ) WeakVocabulary ( ) Easy ( ) Medium ( ) DifficultGrammar ( ) Easy ( ) Medium ( ) Difficult

Rey to Understanding ( ) Vocabulary ( ) Grammar( ) Context ( ) Pragmatics( ) Other:

Level of Difficulty NL NM NH IL IM IH A A+ S=========== ===== ============Stimulus (Polish):

(Translation)

Question:

Options: (* = Key)(A)

(B)

(C)

(D)

= ====== = = ==Author: Date:

12G

Appendix I

Examinee Background Questionnaire

127

!PT FIE111 TIESTINGITT BACKGROUND INFORMATION SHEET

In what land of school do you presently receive Polish language butruction?

1 = Public High School2 = Private High School3 .. Colkge or University4 .. Community School/Weekend language Program5 .. Other

1 How many years of Polish language study, NOT INCLUDING ME CURRENT YEAR,have you completed?

0 .. None1 ail year2 = 2 years

3 3 yews4 = 4 or more years

if How many college Mine credits, NOT INCLUDING YOUR CURRENT COURSE,haw you already earned in Polish?

0 = NoneI = 1 t o 32 = 4 to 63 .. 7 to 94 .. 10 to 12

5 .. 13 to 156 = 16 to 187 = 19 or more8 NE NOT APPLICABLE

17: For how many hours a week does your perm Polish language class meet?

0 = Not Applicabk1 .. 1 hour/week2 = 2 hours/week3 = 3 hours/week4 a. 4 hours/week

0: Are you of Polish ethnic heritage?

0 = Yes1 = No

Do you speak Polish at home?

S .. 5 hours/ week6 = 6 hourshseek7 = 7 hours/weeka = 8 hours/week9 = 9 or more hours/week

0 = Yes, most of the timeI .. Yes, but less than half the time2 = Yes, but very Wrequently3 = No

128 BEST COPY AVAILABLE

Appendix J

Table of Standard Error of Estimation

125

Standard Error of EstimationPolish Proficiency Test

Scaled Scorr Listening _Reading Structurc171 25 26 27170 24 25 27169 23 25 26168 22 24 25167 22 23 24166 21 22 23165 20 22 23164 20 21 22163 19 20 21162 18 20 21

161 18 19 20160 17 19 19

159 17 18 19

158 16 18 18157 16 17 18

156 15 16 17

155 15 16 17

154 14 16 16

153 14 15 16

152 13 15 15

151 13 14 15

150 12 14 14

149 12 13 14148 12 13 13

14- 11 13 13

146 11 12 13

145 11 12 12

144 10 12 12

1431

10 11 12

142 10 11 11

141 10 11 11

140 9 10 11

139 9 10 10

138 9 10 10

13' 9 10 10

136 8 9 9135 8 9 9134 .

8 9 9133 8 9 9132 8 8 9

131 7 8 9130 8 8129 ' 8 8128 -7 8 812' 8 5

126 7 8125 7 7

124 6 ,7

123 6

122 6 7

121 6

120 6

119 6

118 6 6

11" 6 6 t,

130

Standard Error of EstimationPolish Proficiency Test

Scaled Score Listenint R.mdinR Structure116 6 6 6115 6 6 6114 5 6 6

113 5 6 6112 5 6 6111 5 6 6110 5 6 6109 5 5 6108 5 5 6107 5 5 6106 5 5 6105 5 5 6104 5 5 6103 5 5 5

102 5 5 5

101 5 5 5

100 5 5 5

99 5 5 5

98 5 5 597 5 5 5

96 5 5 595 5 5 594 5 5 5

i

93 5 5 592 5 5 591 5 5 590 5 5 5

. 89 5 5 688 5 5 68' 5 5 686 5 5 685 5 5 684 5 5 683 5 5 682 5 5 6

81 5 5 6

80 5 5 679 5 5 678 5 5 677 5 5 676 5 5 6

175 5 574 5 5

'3 5 5

72 5 571 5 5

70 5 5

69 5 5

6S 5 5

6' 5 6

131

Appendix K

Comprehension Tasks for the PPT Items

132

Comprehension Tasks for the PPT Items

Each item in the listening and reading comprehension section was classified by the itemdevelopers as to the type of comprehension task required to answer the item correctly. Threebroad areas of comprehension were used: understanding the main idea, understanding a factor detail, or understanding an inference.

Items identified as requiring the examinee to understand the main idea typically requirethe general comprehension of the entire passage. In same cases, the main gist of the passagemay be grasped with the comprehension of one or two key words or phrases. In the majorit.of the items, however, comprehension of the main idea comes only if most of the passage iscorrectly understood.

Items identified as requiring the examinee to understand a fact or supporting detailgenerally require comprehension of the main idea and some supporting detail. However, insome non-prose items from authentic materials (e.g., advertisements), an item in thiscategory may entail the comprehension of isolated words.

Items identified as requiring the examinee to understand an inference require a thoroughcomprehension of the entire passage in order to identify a correct implication based on thepassage but which is not stated directly in it.

Comprehension Task Listening Section Reading Section

Understand the Main Idea 23 items 20 items

Understand a Fact or' Supporting Detail

20 items 26 items

Understand an Inference 7 items 4 items

1-1 3,

BEST COPY AVAILABLE

Appendix L

Grammatical Areas Covered in the PPT Structure Items

Grammatical Areas Covered in the PPT Structure Items

Each item in the structure section of the PPT was classified as involving primarily eitherone part of speech or another aspect of structure (such as negation). This table shows thenumber of seems involving those primary forms. Specific aspects of each form (such as case,number agreement, gender, tense) are not listed separately.

Form Number of Items

Nouns 11

Verbs 7

Adjective/Adverbs 6

Pronouns 5

Prepositions 3

Other 3

135

Appendix M

Sources for PPT Passages

136

Sources for PPT Passages

This table lists the most

Listening Passages

AdvertisementsCommentariesInterviewsMoviesRadio News ReportsRadio AnnouncementsRecorded ConversationsT. vision News ReportsI :- :.:vision Announcementselevision Programming

Weather Forecasts

widely used types of sources for passages found on the PPT.

Rcading_Zaasago

AdvertisementsBulletinsEditorialsInstructionsLabelsLettersMagazine ArticlesNewspaper ArticlesPopular NovelsSchedulesSigns

Appendix N

PPT Announcement

138

Center forAppliedLinguistics

CAL Announces the Development of the Polish Proficiency Test

With funding from the U.S. Department of Education, the Centerfor Applied Linguistics (CAL) is currently developing a nationalstandardized test in the Polish language. This test, which willinclude listening and reading comprehension sections, will beknown as the Polish Proficiency Test (PPT). When in operation,this test is expected to help strengthen Polish languageinstruction and learning in the United States by serving as atool to both assess student progress across programs and evaluatethe effectiveness of individual programs.

The PPT Development Project, initiated in October, 1991, consistsof three working groups: an External Advisory Committee, a LocalTest Development Committee and a Project Supervisory Group.These three groups include a wide variety of individuals withconsiderable background and stature in the field of Polishlanguage instruction, foreign language education and testing.Dr. Charles W. Stansfield, Director of Division of ForeignLanguage Education and Testing at CAL, serves as the ProjectDirector.

The Polish Proficiency Test is expected to be available by theend of September, 1992. As of December, 1991, it is in theprocess of item development. In February, 1992, the field testversion of the PPT will be administered at different sitesnationwide. In April and May, 1992, the final version will beadministered for norming purposes.

Since the PPT is designed to be a general proficiency test andnot an achievement test, and since it is being developed withauthentic materials, it is hoped that teachers and students inPolish language teaching programs will benefit from participatingin the developmental testing and by using the PPT once it is inoperation. If you need more information about the PolishProficiency Test, or if you would like to have your programparticipate in its testing in February, or in April and May,please contact:

Mr. Xixiang JiangPPT Project Coordinator

Center For Applied Linguistics1118 22nd Street, N.W.Washington D.C. 20037

Telephone:(202)429-9292

Please note that all development testing is done free of chargeto the participating institutions.

13S1118 22nd Street N W Washington D C 20037 (202, 429-9292 Fax (202)659-5641

BEST COPY AVAILABLE