AssessingVocabulary and Writing

Assessing Vocabulary

Dimensions of Word Knowledge

• Accessibility• Morphophonology• Syntax• Semantics• Collocation• Association• Conceptualization• Frequency• Formality• Register

Each dimension represents a continuum

No knowledge Complete knowledge

Commonly used vocabulary tests

Which dimensions of word knowledge do they usually assess?

Do they take into consideration the scalar (continuum) qualities of word knowledge?

Dimensions of Word Knowledge

• Accessibility• Morphophonology• Syntax• Semantics• Collocation• Association• Conceptualization• Frequency• Formality• Register

Vocabulary Size versusVocabulary Strength

Size: the number of words a learner knows Strength (or Depth): how well the learner

knows certain words

Popular tests of vocabulary size

Nation’s Vocabulary Levels Test The Eurocentres Vocabulary Size Test TOEFL Peabody

Popular tests of vocabularysize and strength

The Vocabulary Knowledge Scale The Depth of Vocabulary Knowledge Test Computer Adaptive Test of Size and Strength

Comprehensive Assessment of Vocabulary Knowledge(Based on samples of learners’ speech and writing)

Holistic measures Primary trait measures Rubrics Analytical measures

Comprehensive Analytical Assessment of Vocabulary Knowledge and Use

Tokens Types Lexical Density Lexical Sophistication Lexical Frequency Profile Lexical Diversity Lexical Error Indices Lexical Choice Analysis Lexical Style/Lexical Signatures

Assessing Writing

Popular approaches to assessing writing abilities

Holistic assessment Primary trait assessment Rubric assessment Analytical assessment Portfolio assessment

– Showcase portfolios– Collections portfolios– Assessment portfolios

Which approaches have these advantages?

ADVANTAGESA. Indicates to what extent students are benefiting from instruction.B. Even with minimal rater training can produce high reliability.C. Gives a very direct measure of writing effectiveness, using a simple protocol.D. Provides the most detailed feedback to writers.E. Is the most learner-centered.F. Is best for diagnosing grammatical and mechanical needs.G. Is best for diagnosing discourse-level, content, and organizational needs.H. Is the most practical for proficiency-rating and placement-level purposes.I. Most directly links assessment with instruction.J. Encourages students to reflect on their work, analyze progress, and set

improvement goals.K. Encourages students to produce writing that is worth sharing and retaining for

review.

Which approaches have these disadvantages?

DISADVANTAGESA. Provides the writer with little or no feedback.B. Can make the teacher feel that she is giving up control of the assessment process.C. Provides feedback that may steer the writer’s attention to areas of writing that are

least important to the overall effectiveness of her text.D. Relies on teacher-selected criteria that may be either over-representative or under-

representative of the components of good writing.E. Requires multiple trained raters in order to achieve adequate reliability.F. Can be extremely time-consuming.G. Requires raters to evaluate writing in a way that real-world readers do not do.H. Relies on several rating criteria that should presumably be weighted differently, but

weighting criteria is difficult and subjective.I. Is least likely to lead directly to improvements in writing.J. Provides the least useful assessment information to teachers.K. Provides the least useful assessment information to writers.L. Is the most subjective.M. Provides the most confusing assessment results.N. Is the least pedagogically sound.O. Is the least criterion-referenced.

Principles of Testing Writing

Set writing tasks that are properly representative of the tasks that learners should be able to perform.

The tasks should truly represent learners’ writing ability.– Use as many tasks as you can.– Test only writing ability.– Restrict options.

Principles of Testing Writing (cont.)

The tasks must be scored reliably.– Elicit sufficiently long samples.– Establish a clear set of criteria for scoring

Whether holistic, primary-trait, rubric, or analytic Use 2-4 raters. Train the raters. Monitor raters’ performance.

Some new directions in assessing writing

Self evaluation Peer evaluation Picture and film stimuli Writing for authentic purposes Process evaluation and progress profiles

Some realistic day-to-day writing tasks

Writing checksPaying billsTaking notes and phone messagesWriting reminder notes to oneselfWriting a shopping listWriting a schedule/itinerary for oneselfWriting letters/e-mail messages to friends and relativesWriting letters to people/companies you do business with

-inquiries-complaints-appreciation

Filling out forms-credit card applications and the like-medical background forms for healthcare providers-personal background forms for potential employers

Writing a classified ad for a newspaperWriting a letter to the editorWriting a curriculum vitae or resumeWriting a cover letter for a job application

How do you assess the writing abilities of low-level learners?

realistic tasks (tasks they are truly likely to encounter)

structured/discrete-point test items