On Scoring Guides

everything you were afraid to ask

PART TWO

Critical thinking as“distributed compentency”?Generalized thinking (e.g., Bloom) explained to others in the institution?Generalized thinking explained to laypersons

outside the institution?Integrative thinking?Thinking in the major or discipline?Thinking in a specific community?Some combination?Other

St. Olaf E-Portfolio Thinking

Reflective Thinking

Integrative Thinking

Thinking in Context

Thinking in Community

Specific and Flexible

WPA Outcomes Statement for First-Year Composition

•Rhetorical Knowledge•Critical Thinking, Reading, and Writing•Processes•Knowledge of Conventions

Rhetorical Knowledge

By the end of first year composition, students should

--Focus on a purpose --Respond to the needs of different audiences--Respond appropriately to different kinds of

rhetorical situations --Use conventions of format and structure

appropriate to the rhetorical situation --Adopt appropriate voice, tone, and level of

formality --Understand how genres shape reading and

writing --Write in several genres

Rhetorical Knowledgefor ASU CompositionOur writing courses will focus on helping students develop and use a rhetorical framework to analyze writing situations, in a number of ways. Students will learn how to--use heuristics to analyze places, histories, and cultures --be aware of the components of argument and create their own

arguments in conversation with other members of their discourse communities

--synthesize and analyze multiple points of view --use a variety of argumentative strategies to write for a variety

of audiences --express a working knowledge of key rhetorical features, such

as audience, situation, and the use of appropriate argument strategies --adopt appropriate voice, tone, and level of formality --use conventions of format, structure, and language appropriate to the purpose of the written texts --be able to focus on a specific rhetorical purpose

So . . .

PLAN

COLLECT DATAUSE RESULTS (aka, feedback loop)

UNC Charlotte English Education

*Portfolio*Interviewto assess readiness to student teach

*What is your favorite reading?*What African American literature

would you like to teach? What Native American literature would you like to teach?

Issues . . .

*why were students preferring shorter texts?

*why could they not identify more than a single text in a given field? Was this a problem?

*how did they perform on national tests?

*what feedback did we get from schools?

CHANGES

• Added a course in Native American Lit

• Added a course in African American Lit

• Required a course in Ethnic Literature

• Encouraged all faculty to use “real” texts

RESULTS? new issues!!

PORTLAND STATE CRITICAL THINKING:program design & METHOD

A random sampling of these portfolios that was stratified for each class formed the analysis set. Portfolios were selected using a random number generator and a numbered list of students from each Freshman Inquiry class. Five student names and one alternate (and instructions for choosing a second alternate, if necessary) were given to each instructor. Alternate names were used if one or more of the original five students chosen appeared on the class list, but did not complete the course.

~~interdisciplinary, team-taught first year seminar~~

The scoring guides (rubrics) used in the Summer Portfolio Review were internallydeveloped. A previous attempt to use an externally developed rubric for critical thinkingwas not successful because the rubric was not contextually relevant to the PSU student work.

The new rubric for critical thinking was not completely developed by the time ofthis summer’s review, and the review itself stood as a development process for thisrubric.

PORTLAND STATE AND CRITICAL THINKING

WHAT HAPPENED?

Following the June 2000 review of Freshman

Inquiry portfolios, each Freshman Inquiry

team was required to review information from

the portfolio review and, if available, from

the end-of- year course evaluations. Each team

reported to the Freshman Inquiry Coordinator,

specific, planned course revisions based on the

asessment information.

In the summer 2000 meeting, the “Metamorphosis” team decided to focus on the critical thinking goal. To this end, they instituted a major re-working of the fall term course,revising the texts and focusing more clearly on central concepts. They also added morescience content in the first and second quarter courses. This team raised questions aboutthe critical thinking rubric, which is scheduled for revision this year.

GET SPECIFIC! WHAT DOES THIS MEAN?

Did you say technology?

What some of this means for guides . . . • Scoring Guides Can Show Development, Achievement, or

Both--What is the guide intended to document, and why?

• Text and Context: Scoring Guides Don’t Act Alone.--How will you introduce them to students? And for what

purpose/s?

• Scoring Guides Signal the Philosophy of the Institution.

--Is the model of learning showcased in the guide a deficit model of learning (think error avoidance or removal) or an asset inventory model (think building on strengths)?

**Scoring Guides Inherently Enact a Visual Rhetoric --Most scoring guides list a set of elements organized into

an analytical framework or a holistic framework, suggesting that each item is weighted equally. When the guide is put into an institutional context and/or is used to rate student work, often some items are more valuable than others. Which items are the more valuable, and why, and how can that be signaled within the guide itself?

**Scoring Guides, as Genre, Change--The scoring guide we would have created for writing twenty

years ago would exclude much of what we do in writing today; critical thinking is likewise defined more ambitiously today than in earlier times. The scoring guide we create today should do the best job it can do today. It will need to be changed tomorrow.

How to begin?

*Gather some student work.(What counts as work?)

*Gather some model scoring guides.*Get some food.

**Talk! **Identify what you like—that’s vocabulary. Consume? Produce?Achievement? Development? **Identify a structure.**Give it a go, and call it a pilot.

You’re never really ready . . .

It’s Better to Start Small,

but to Start . . .

Call it what you will—a pilot, a field test—start! Start small. Get a taste of success, and build on that.