ITEM & TEST ANALYSIS

Universidad Pedaggica Experimental LibertadorInstituto Pedaggico de Caracas Departamento de Idiomas ModernosCtedra de Lingstica Evaluacin ILEANALYSIS OF TEST FORMATS Autores:Barrientos AurimarGranado ReinaNoriega AnahyRamrez Dayana

Caracas, October 20131.-) MATCHING: This type of test has two sets of items to be matched with each other. Example: Match with a line the phrases on the left with the correct tense on the right. (3 points = 1 point each)

A Ba. verbs ending in s or es a. present progressiveb. verbs ending in ed or d b. simple present.c. verbs ending in ing c. present perfect d. simple past

Types of test formats2-) GAP FILLING: Is a test in which words are removed from a test and replaced with spaces. Example: Choose the word which best completes the sentences. (1 point)

1.-) the long dark days and lack of a job made him feel __________.a-) alarmed b-) excited c-) depressed d-) satisfied.

Types of test formats3-) CLOZE TEST: It consist of a portion of text with certain words removed, where the participant is asked to find the missing words.

Example: - Complete the text with the correct words. (3 points= 1 point each)

Today I went to the ____________ and bought some milk and eggs. I knew it was going to rain, but I forgot to take my ________, and ended up getting wet on the way ____________. Types of test formats4-) COMPLETION TEST: it requests that the learner complete the sentences.

Example: - Choose the answer choice that contains the word or words that best complete the sentences. (3 points= 1 point each)

a- Rang b- Studying mathc- Teaching French

1.-) the bell ________________________2.-) The teacher was _________________3.-) The students were _______________

Types of test formats5-) MULTIPLE CHOICE TEST: Is a test with a number of set answers for each question, and the learner must choose which group of answers is correct. Example: - In the simple past tense what is the auxiliary used to structure interrogative and negative sentences? (3 points= 0,50 points each)a-) do__________b-) does________c-) dont _______d-) did_________e-) doesnt _____f-) didnt _______

Types of test formats6-) TRUE-FALSE: Is a test consisting of a series of statements to be marked as true or false.

Example:1-) Mark with an x true or false in the following question. (1 point)

a-) the sentence the bell rang and the students didnt come to the classroom is a simple past tense. True _______ False _____

Types of test formats7-) TRANSFORMATION: Is a test where the learner change a sentence or word in another.

Example:Change the sentence from affirmative to negative. (2 points= 1 point each)

a-) They have stamped the letter.____________________________________________

2-) He escaped from one prison to another.___________________________________________

Types of test formats8- ) QUESTIONS TEST : It asks information or is used to test someones knowledge.

Example:- Answer the following questions. (2 points= 1 point each).

a.) Which resonance chamber is completely fixed in dimensions and shape?

b-) What are two key articulatory features of nasal sounds?

Types of test formatsDifferences between Cloze test, Gap filling and completion testCLOZE TESTGAP FILLING TESTCOMPLETION TESTCloze test involves a piece of text from which a number of words has been removed. It requires the ability to understand context and vocabulary. This exercise is applied for the assessment of native and second language learning. Cloze test does not have a bank of word.Gap filling tests are constructed over a single isolated sentence with a bank of words and it is applied to evaluated vocabulary.Example: With her red hair and lively, outgoing manner, she was a ______charactera-) ugly b-) tearful c-) flamboyant d-) sectretiveCompletion test requires the student to answer a question or finish an incomplete statement by filling in a blank with the correct word or phrase. It can measure cognitive ability.Example: The first Prime Minister of Canada was _________________.

Key:Sir John A. Macdonald Strengths and WeaknessesMatchingGap-filling

StrengthsWeaknessesObjectiveEasy to designEasy to include a representative example.Easy to score.Not good for higher levels of learning. Only for testing grammar and vocabulary.Time consuming for students.StrengthsWeaknessesObjective.Can cover wide range o content.Easy to correct.Must be constructed carefully Difficult to design an item with only one possible answer.Strengths and WeaknessesStrengthsWeaknessesEasy to design. Easy for the students to understand.Easy to correct. Carefully selection of texts.Unexpected correct answers are possibleScoring can be trickyCloze Test

StrengthsWeaknessesEasy to design.It is good to evaluate listening activities.Requires simple recognition of the informationIt can be irritating for students. Difficult to grade depending on the evaluation criteriaDepend on knowledge of language.Completion

StrengthsWeaknessesCan cover a broad range of difficulty levels.Usually requires less time for tests takers to answer.Can cover a broad range of content.Test takers may perceive questions to be tricky.Emphasizes on guessing rather than in productionDifficult to design.

Multiple Choice

Strengths and WeaknessesStrengthsWeaknessesCan cover a broad range of difficulty levels. Usually requires less time for tests takers to answer.Easy to design.Encourages guessing do to 50/50 chance of being correct.Does not allowed test takers to demonstrate broad range of knowledge.True-False

StrengthsWeaknessesEffective for testing grammar.Allow recognition of a connection between Grammar and Meaning.Easy to design.Test of understanding is very artificial.It can be more than one possible transformation in some cases.Transformation

Strengths and WeaknessesStrengthsWeaknessesMeasures higher levels, cognitive learning and overall comprehension. It is not easy to score.It takes more time to correct answer.QuestionsPoor and Good itemsMatchingA set of directions: Match column A with column BIt should be: On the line next to each description in Column A, place the letter of the president in Column B whom it describes. Answers in Column B may be used only once. Column A Column B1- George Washington A- The first President of the USA2- Abraham Lincoln B- He used to be a professional dancer3- Richard Nixon C- Was well known for his humanitarian causes4- Jimmy Carter D- He used to play golf5- Ronald Reagan E- Was assassinated in a theater F- He was a prime minister before being presidentA column of statements: A column of responses: The instruction do not clarify how many times items in column B may be usedContain the same number of items, so the last item could be answered by eliminationItems are too heterogeneous, making the answers too obvious Recomendations for writing Good Matching itemsGive Clear InstructionsKeep the two sets of items homogeneousCreate more responses than premisesBe conscious of layout and formatDoNOTlist premises in the same order as responsesThere shouldNOTbe keywords appearing in bothIt shouldNOTbe possible to subdivide both into two or more discrete subsetsPoor and Good itemsPoor and Good itemsIncomplete StatementA __________ is a type of approximant, pronounced like a vowel but with the tongue closer to the roof of the mouth. Options: a- Alliteration b- Phoneme c- Glide d- Approximant e- SemivowelThey do not belong to the same category Avoid grammatical clueGap fillingPoor and Good itemsRecomendations for writing Good Matching itemsThe gap should contain the key information being tested

Make sure the meaning of the statement is not lost

Make sure one correct answer is possible

Avoid grammatical cluesPoor and Good itemsClozeIncomplete text____________is an ______________field that ________the_______mechanisms in thehuman ________that control the ____________, production, and _____________ oflanguage. Do not delete the first part of a text if you are omitting the key words Provide sufficient and relevant information to restore the missing wordsIt should be: Neurolinguistics is an interdisciplinary ________ that studies the_______ mechanisms in thehuman ________that control the ____________, production, and _____________ oflanguage. Poor and Good itemsRecomendations for writing Good Cloze itemsOmit every 5th word, replacing it with a blank space for the student to write in the answer.

Instruct students to write only one word in each blank and to try to fill in every blank.

Includecontext languageandcontent terms.Poor and Good itemsCompletionIncomplete statementAccording to________, personality is made up of _______major systems, the_Id_, the ________ and the __________________Do not omit so many words from the statementOmit only significant words from the statementUse blanks of the same length It should be:According to Freud, personality is made up of three major systems, the _________, the ________ and the _____Recomendations for writing Good Completion itemsAvoid the statement becomes ambiguous.

Use blanks of the same length throughout the test so that the length is not a clue.

If possible, put the blank at the end of a statement rather than at the beginning.

Limit the required response to a single word or phrase.Poor and Good itemsPoor and Good itemsMultiple ChoiceStemOsteoporosis has haunted women since the dawn of history. Which of the following is not a symptom of osteoporosis?Avoid irrelevant materialPresent a single statement a. decreased bone densityb. frequent bone fracturesc. raised body temperatured. lower back painDistracters The keyIt should be: Which of the following is a symptom of osteoporosis? a. hair loss b. raised body temperature c. decreased bone densityRecomendations for writing Good Multiple Choices itemsPut as much of the question in the stem as possibleUse only plausible and attractive alternatives as distractors.Do not create distracters that are so close to the correct answerAvoid the choices All of the above and None of the abovePoor and Good itemsPoor and Good itemsTrue-False Two possible alternativesA poem, an arrangement of words containing meaning and musicality, could be usually referred to as an English sonnet: abab cdcd efef gg.

True____ False____Avoid unnecessary materialAvoid this type of wordsIt should be: A poem could be referred to as an English sonnet: abab cdcd efef gg. True____ False____Recomendations for writing Good True-False itemsInclude only one main idea in each item.Use statements which are unequivocally true or false.Avoid the use of words which would signal the correct response:Words that indicates the response is false: none, never, always, all and impossible

Words that indicates the response is true: usually, generally, sometimes and oftenPoor and Good itemsPoor and Good items Transformation A statement Change the sentences to the future simple:Several responses to transform- Jenny studied English yesterday.

-Jenny went to the movies with her family. -He jogs in the park.

-They enjoy the day in the mall. The activity lacks context The sentences should be provided in the same tense, to prevent confusion This exercise requires more contextualized instructions, such as: This is what Jenny usually does on the weekend; write about what she is going to do next weekend.Poor and Good itemsQuestion A questionWhat is Psycholinguistics? Is the study of thepsychologicalandneurobiologicalfactors that enablehumansto acquire, use, comprehend and producelanguage.How long have you been working at the IPC?Ive been working here since 2002. Responses depend on each question. Answers could be long or shortItem analysis:

- It measures the performance of each item of a test.

- It help teachers to revise and improve test and items as a whole.

- Reliability and validity are built into a test.

- Give teachers more diagnosis information on students.ITEM & TEST ANALYSISITEM & TEST ANALYSISElements on item analysis:

- To exam the difficult level of items.

- To determine the discrimination power of each item.

- To interpret the effectiveness of distractors on multiple-choice and matching items.

Index of difficulty = IF:

It represents the percentage of students answering correctly the item in a test. Best answers are neither too difficult nor too hard.

IF = # students answering the item correctly Total students reponding the test What does IF means? High scores, Median and Low scoresITEM & TEST ANALYSISIndex of discrimination= IDIt reflects how effectively an item defferentiates on a performance of students who did well on a test. ID = (CU CL) NSGCU = # correct responses of the upper groupCL = # correct responses of the lower groupNSG = # of students per group

ITEM & TEST ANALYSISITEM & TEST ANALYSISExample: Assess of Item No. 23: Scores

IF = 9 = 0,45 x 100 = 45% Item Difficulty 20 = Its the percentage of students answering correctly the item

ITEM & TEST ANALYSISID= (7-2) = 5 = 0,55 X 100 = 55% 9 9

Item Discrimination:

It represents the percentage of high-scoring individuals responding correctly versus the number of low-scoring individual responding correctly to an item.Distractor efficiency:It represents the way distractors lure on test- takers, by guessing, especially when they are not prepared. Distractors are useful on multiple-choice and matching items.

ITEM & TEST ANALYSISITEM & TEST ANALYSIS-To assess responses to distractors, teachers can use a frequency table. It tells you the number and percentage of test-takers who select the distractor.

-Distractors selected by a few students should be replaced, because they are so implausible to students that anyone selects them.

-Distractors should be selected by at least a few students.

-One distractor selected by many students may indicates a confusing option.Distractor analysis:

Teachers must be careful with the items analysis because the indexes cannot be applied on informal evaluation (essays), and the collection of data demand accuracy.Conclusions and recomendations

The difference between school and life? In school, you're taught a lesson and then given a test. In life, you're given a test that teaches you a lesson. Tom Bodett Thanks for your attentionBrown, D. (2004) Language assessment: Principles and Classroom practices. Longman, London.

Cunningham, G. (1998) Assessment in the Classroom: Constructing and interpreting Tests. Rutledge, London.

Morrow, J. and others (2010) Measurement and Evaluation in Human Performance and Human kinetics, Illinois.

Madsen, H. (1983) Techniques in testing. Oxford University Press, New York.References