Upload
maury-martinez
View
1.199
Download
0
Embed Size (px)
DESCRIPTION
Testing
Citation preview
STAGES OF TEST DEVELOPMENT
&COMMON TEST TECHNIQUES
DANIEL F. CALI. O.
STAGES OF TEST
DEVELOPMENT
STATING THE PROBLEM
SPECIFICATIONS
WRITING AND MODERATING
ITEMS
ITEMS ON NATIVE
SPEAKERS
ON NON-NATIVE
SPEAKERS
ANALYSIS OF RESULTS
CALIBRATION OF SCALESVALIDATION
HANDBOOKS FOR TEST TAKERS
TRAINING STAFF
OUTLINESTAGES OF TEST DEVELOPMENT
• SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM?
TEAM!
• IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF…
…OBJECTIVITY
…CRITICISMS
…NATIVE COMMAND OF THE LANGUAGE
STEPS…
STATING THE PROBLEM
QUESTIONS
KIND?
PURPOSE?
ABILITIES?
HOW DETAILED?
HOW ACCURATE?
WASHBACK?
CONSTRAINTS?
FINAL
PROFICIENCY
PROGRESS
DIAGNOSTICPLACEMENT
RESULTS
EXPERTISE
FACILITIES
TIME
• ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO:
GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR SITUATIONS!
STATING THE PROBLEM
WRITING SPECIFICATIONS FOR THE TEST
• A) CONTENT
• B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• C) CRITERIAL LEVELS OF PERFORMANCE
• D) SCORING PROCEDURES
A) CONTENT
• INCLUDE CLEAR SPECIFICATIONS REGARDING:
• SKILLS (SUB-SKILLS)
• TYPE OF TEXTS
• ADDRESSEES
• LENGTH OF TEXTS
• TOPICS
• (READABILITY)
• STRUCTURAL RANGE
• VOCABULARY RANGE
• DIALECT, ACCENT AND STYLE
• SPEED OF PROCESSING
B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• TEST STRUCTURE
• NUMBER OF ITEMS
• NUMBER OF PASSAGES
• TIMING
• MEDIUM
• KIND OF TEST TECHNIQUE(S)
C) CRITERIAL LEVELS OF PERFORMANCE
• ACCURACY
• APPROPRIACY
• RANGE
• FLEXIBILITY
• SIZE
D) SCORING PROCEDURES
• WHAT RATING SCALE WILL BE USED?
• HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK?
• WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF WORK?
WRITING AND MODERATING ITEMS
• A) SAMPLING
• B) WRITING ITEMS
• C) MODERATING ITEMS
A) SAMPLING
HOW THE TEXTS ARE GOING TO BE CHOSEN?
B) WRITING ITEMS
• TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS!
• AN ITEM WITHOUT A KEY IS INCOMPLETE!
• “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR ABANDONED IS THROUGH THE PROCESS OF MODERATION”.
C) MODERATING ITEMS• INTERVATION OF TWO COLLEAGUES:
INFORMAL TRIALLING OF ITEMS ON NATIVE SPEAKERS
• 20 OR MORE…
• THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF:
AGE
EDUCATION
GENERAL BACKGROUND
TRIALLING OF THE TEST ON A GROUP OF NON-NATIVE SPEAKERS
SIMILAR TO THOSE FOR WHOM THE TEST IS INTENDED
• PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.
ANALYSIS OF RESULTS OF THE TRIAL; MAKING OF ANY NECESSARY CHANGES
• STATISTICAL AND QUALITATIVE ANALYSIS.
HOW DIFFICULT ARE THE ITEMS?
DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)
CALIBRATION OF SCALES
• IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH OF THEM TO A POINT ON THE RELEVANT SCALE.
VALIDATION
LOW STAKES
TEST
HIGH STAKES
TEST
TOEFL
POP QUIZE
S
WRITING HANDBOOKS FOR TEST TAKERS, TEST USERS AND STAFF
RATIONALE DEVELOPMENT DESCRIPTION SAMPLE ITEMS
ADVICES ON PREPARING FOR
TAKING THE TEST
TEST SCORES INTERPRETATIO
NS
TRAINING MATERIALS
TEST ADMINISTRATIO
N
TRAINING STAFF
NEED TO BE TRAINE
D
INTERVIEWERS
RATERS
SCORERSCOMPUTER OPERATORS
PROCTORS
COMMON TEST
TECHNIQUES
OUTLINE
COMMON TEST
TECHNIQUES
GAP FILLING
MULTIPLE CHOICE
TRUE OR
FALSE
SHORT-ANSWERS
DEFINITION
WHAT ARE TEST TECHNIQUES?
• MEANS OF ELICITING BEHAVIOR
RELIABLE AND VALIDBEHAVIOR
BEHAVIOR WHICH CAN BE RELIABLY
SCORED
LESS TIME AND
EFFORT
BENEFITIAL WASHBACK
TECHNIQUES
MULTIPLE CHOICE ITEMS
• STEM
• DISTRACTOR
MULTIPLE CHOICE ITEMS
ADVANTAGES DISADVANTAGES
WASHBACK
LACK OF DISTRACTORS
GUESSING
PERFORMANCE
TESTING OF RECEPTIVE
SKILLS
RELIABLE SCORING
DIFFICULT TO WRITE ITEMS
GREAT DEMAND ON TIME AND EXPERTISE
CHEATING
YES/NO AND TRUE/FALSE ITEMS
MULTIPLE CHOICE (2 ITEMS)
TOO INFORMAL
REASONS?Reliability and validity
SHORT-ANSWER ITEMS
READING TEST
S
LISTENIN
G TEST
S
GAP FILLING ITEMS
• IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED!
• A: WHAT WILL HE DO?
• B: I THINK HE ______ RESIGN.
WILL, MIGHT, COULD MAY, ETC.
BUT:
A: I WONDER WHO THAT IS.B: IT ______ BE THE DOCTOR.A: HOW CAN YOU BE SO CERTAIN?
• Now you know what a teacher should do when preparing tests, what advices would you give?
• (Each one should only mention only 1 advice)
Names Advices
Carolina Think of the applicability to real situations ar real life situations.
Paola Put in your students´ shoes.
Mauricio Consider multiple intelligences.
THANK
YOU!
REFERENCES
• Huges, Arthur. Testing for Language Teachers. New York: Cambridge UP, 2003.