COURSE ASSESSMENT

“Using Disciplinary and Interdisciplinary Strategies for Assessing Student Learning”

Dr. Anita Helle

Assessing the “Ineffable”

The Teagle Foundation:(http://www.teaglefoundation.org)

While assessment is changing the landscape of higher education, specialists in many fields believe that at the heart of their disciplines, is knowledge that is difficult to assess . . . we’ve moved into a more nuanced conversation that is seeking out ways to make difficult-to-assess knowledge integral to our colleges and universities and to ways our faculty think about their work. --Bob Connor (Feb 2011)

Examples of “difficult-to-assess” knowledge

critical thinking analytical reasoning• moral reasoning• “deep” reading• civic engagement• creativity

T.S. Eliot, “Little Gidding” (1942) – “So at the end of our exploring, we shall arrive at where we started, and know the place for the first time.”

Inquiry-focused, interactive models and approaches

Especially for literature faculty who a) teach writing and b) are trained to negotiate thecomplex demands of aesthetic discourse, the spitting forth of received knowledge hasrarely been a viable or desirable pedagogical stance. Given our historical relationship to the aesthetic, literature faculty are, for better or worse, structurally positioned to recognize that the very definition of the “student-centered university” represents a radically simplified and perhaps unnecessarily divisive notion of the pedagogical task.

Confronting the issue of faculty motivation is certainly the most difficult task oassessment,” as Peggy Maki acknowledges in Assessing for Learning: Building a SSustainable Commitment Across the Institution. She writes that “directing professional energy and curiosity into what and how students learn … is an essential process in a learning centered institution” (11).

Peggy Maki, Assessing for Learning: Building a Sustainable Commitment Acrossthe Institution (Stylus Publishing), 2nd Edition (December 2010).

What qualities do we value?

Rubric creatio

n

Refining evidenc

e:teachin

g

Refining evidenc

e: faculty reflecti

on

Curricular

decisions

A gateway course for the English Major: ENG 345 Introduction to Literary Criticism and Theory

ENG 345 Introduction to Literary Criticism and Theory

Study and analysis of the critical frameworks and methodologies for the

interpretation of literature and culture. (current catalog description)

“Everything depends on the assumption that students are entering a

discipline. They have to learn the codes. But any artisan knows you can be

creative with codes; otherwise all chairs would look the same. The more

experienced member of the discipline will recognize the codes at work,

although he or she may not always be able to show this to someone who is

not in the discipline.”

Faculty Survey, 2005

ENG 345: a bit of history – 2005-07assessing the ineffable in literary theory

• Few faculty agreed on what the focus of the key disciplinary course should be, but most faculty agreed that a spate of hiring in the 1980s and 90s had led a more theory-driven version of the major and a need to provide grounding for different interpretive approaches.

• Students tended to view the course as a one-off “showcase” for faculty demonstrations of singular critical approaches, or approaches used in isolation from each other. Since theoretical understanding was treated as “tacit” knowledge in other courses, students struggled to make connection across the program.

• Performances of writing are required for all courses, but the relationship of essays grounded in theoretical knowledge the field and broader applications to discourses and communities of

practice was neither articulated nor easily assessed.

Elements of Disciplinary “Code”

Key words in the discipline of literary criticism and the concepts they denote

Major figures in literary theory and their contributions to modern and contemporary criticism

Theoretical premises of at least four contemporary theories of reading - gender criticism, new historicism, deconstruction, eco-criticism

Encodings - Holistic performances of a written discourse we call “literary and cultural criticism” in a capstone assignment that applies one theory and its critical propositions to analysis of a single literary text.

Measuring Learning: “Authentic” Assessment

Assessment is not, like other forms of high

stakes testing, to sort and rank. Rather, its

purpose is to provide schools with information about how much

students learn — or fail to learn — in their majors, and

over the course of their careers.

--Howard Gardner

Measuring Learning: “Authentic” Assessment

“Traditional” -----------------------------------”Authentic”

Selecting a Response ------------------------------------ Performing a TaskContrived --------------------------------------------------- Real-lifeRecall/Recognition ----------------------------------------Construction/ApplicationTeacher-structured ----------------------------------------Student-structuredIndirect Evidence -------------------------------------------Direct Evidence

Gathering “Authentic” Evidence of Student Learning

Multiple-choice tests-----------------------------------------Writing Rubrics Portfolios Self-reflection

Norm-referenced----------------------------------------------Values-Referenced

Rubrics for Writing Assignments: Function and Design

A well-designed rubric is an effective communication tool. It emphasizes the important skills or concepts to demonstrate. It provides criteria for evaluation and takes the intangible on an unfamiliar assignment and makes it more tangible.

A rubric streamlines the grading process. It quantifies the elusive expectations and makes them clear. There can be no claims from students saying, “You never told us that was needed.” One cannot argue with the on-screen, printed standards. It makes estimates more scientific and grading fairer.

A well-designed rubric for student writing may leave room for instructor judgment, but as a schema, it communicates impartiality to students. A student either meets the defined objectives or does not. This helps promote fairness and increases satisfaction, since there is no preferential treatment when everyone is measured using the same benchmarks.

The variety of rubrics available creates flexibility to meet a wide range of assignments. Holistic rubrics identify all factors for an assignment using a checklist or description. Analytical rubrics provide scales and a set of scores for multiple criteria (2008).

Writing Rubric for ENG 345 Course Assessment (final paper)

The paper as a whole should demonstrate a clear understanding of the central concepts and approach (method) of the theoretical model or key concept.

The body of the paper should carefully explore specific ways the concept or critical framework can be applied to a specific text.

The paper should state and who the implications of critical propositions for analyzing the close reading of particular passage (that is, “If X is taken as a critical premise, one implication is Y….”).

The paper may include strengths and limitations of this approach when applied to the primary text (what is revealed? What could be overlooked or overemphasized?

The writing should include a thesis statement, effective selection and analysis of specific passage in support of an argument, MLA style with in-text citation. (NB: Students are given a set of primary texts and critical theories/concepts to apply.)

Back to Inquiry: Moving from Course Assessment to Program Assessment

Overall Program Outcomes:1) Write effective arguments about a variety of literary and cultural texts.

2) Use information literacy and new technologies to plan and conduct research appropriate to initial and advanced study in English.

3) Recognize and interpret a wide variety of texts and genres, using a range

of theoretical and interpretive strategies, including close reading.

4) Demonstrate the role of context(s) in production, reception, and transmission of literary and cultural texts (across periods, histories, geographical/national spaces, and cultural differences).

What have we learned?

One course alone—despite its foundational weight—should NOT bear the burden of measuring all program outcomes.

Curricular decision taken in 2010-11: An intro-level pre

test understanding critical contexts will be administered

to majors in an entry-level literature survey (ENG 204).

Responding to changes in our field and profession

Filling in gaps – making new connections.

Given the recursive nature of difficult-to-assess knowledge (as Gertrude Stein said about writing, “with practice you get better”), engaged assessment can tell us where there are gaps, where we need to make and to reinforce salient connections across the English program.

Curricular decision taken: create unifying methodologies in a suite of

courses across the 300-level to give additional experience in practical

criticism and respond to changes in critical methodologies.

Ongoing challenges to confronting the “ineffable” and moving forward with a clear focus.

Recognizing that disciplinary majors have more shared goals than we realize.

Using disciplinary models (such as writing) to assess “deep reading” across disciplines.

Building commitment to moving forward where “perfection” in normed measurements and quantification is not within our grasp.

Taking the long view—assessment as a form of advocacy for student learning and engaged evidence.