Classroom Assessment Across the Disciplines

PART THREE

New Applications, New Considerations

Working together, three faculty members from different disciplines seethe similarities and differences in their classes through the use of CATs.

Classroom Assessment Across theDisciplines

Regina Eisenbach, Vicki Golich, Renee Curry

We may compare teaching to selling commodities. No one can sellunless someone buys . . . [yet] there are teachers who think theyhave done a good day’s teaching irrespective of what pupils havelearned.

—Dewey, 1933, p. 35.

Funny, though the twenty-first century dawns, not much has changed for col-lege and university faculty. We still confront the challenge of delivering com-prehensive course content within arbitrarily limited time frames. Most of usgraduate from programs that label research “the scholarship of discovery”(Boyer, 1990) and value it above all else. Then we move on to work in insti-tutions of higher education that perpetuate this focus. Though few of theseinstitutions would exist or survive without students, they seldom regard orreward the difficult and complex art and “scholarship of teaching” as highly asthe scholarship of discovery. Consequently, many of us concentrate on trans-mitting vast amounts of information to our students using the conventionalpedagogy of the lecture. Many remain convinced that “teaching is telling,knowledge is facts, and learning is recall” (Cohen, 1989).

As if these traditional obstacles to teaching were not enough, theapproaching century adds some new, significant challenges to pedagogical suc-cess. First, the explosion of disciplinary information and theoretical thinkinggreatly complicates the selection of course materials. Second, institutionscharge faculty with demonstrably improving inadequate learning outcomes(Ehrmann, 1995, p. 6) in the face of expanding diverse student populations

NEW DIRECTIONS FOR TEACHING AND LEARNING, no. 75, Fall 1998 © Jossey-Bass Publishers 59

60 CLASSROOM ASSESSMENT AND RESEARCH: AN UPDATE

and shrinking financial support. Employers seek skilled and knowledgeablegraduates prepared to solve increasingly complex problems in various workenvironments. To deliver this “product,” faculty must adopt pedagogies thatempower students to identify the tools and information appropriate for theproblem and enable students to assume roles as both effective leaders and teamplayers (Ehrmann, 1995, p. 12).

Finally, research continues to demonstrate that the lecture—while an effi-cient mode for transferring information in the short term—is ineffective if wewant students to remember that information over time. Students forget asmuch as 50 percent of course content within a few months of taking lecture-based courses (Stice, 1987). Current national trends that recognize the impor-tance of excellent teaching have not been accompanied by professionaldevelopment support. No wonder many of us feel overwhelmed.

Fortunately, Classroom Assessment Techniques (CATs) offer a relativelysimple means to meet and overcome these significant challenges. By helpingprofessors “obtain useful feedback on what, how much, and how well theirstudents are learning,” CATs enable faculty to refocus their teaching so studentscan make their learning more efficient and more effective” (Angelo and Cross,1993, p. 3). In short, CATs help faculty concentrate on how, what, and at whatpace students learn key concepts, assumptions, and analytical frameworks thatinform every discipline.

Classroom Assessment and Effective Pedagogy

For years researchers have agreed on at least three principles of good practicein the teaching and learning process: active learning, frequent feedback on per-formance, and frequent student-faculty contact (see, for example, Dewey, 1933;Chickering and Gamson, 1987; Ewell and Jones, 1991). In addition, a sub-stantial body of literature links pedagogies that support these principles to theenhancement of learning (see, for example, Association of American Colleges,1985; McKeachie, Pintrich, Lin, and Smith, 1986; Pascarella and Terenzini,1991). CATs—by design and by measured results—embody these principles:they require active student participation and create a viable, continuous feed-back loop between faculty and students, thereby satisfying the need for fre-quent feedback and student-faculty contact. Together, these activities helpstudents take responsibility for their learning and at the same time enhancethe quality of student learning. CATs have the added benefit of adaptability:faculty can and should adapt them to the particular needs and characteristicsof their students, their disciplines, and their own preferred teaching styles(Angelo and Cross, 1993, pp. 4–6).

Background to the Project

California State University at San Marcos (CSUSM) established a Faculty Cen-ter in the spring semester of 1995 in response to faculty demand for support

61CLASSROOM ASSESSMENT ACROSS THE DISCIPLINES

of their multiple roles. Vicki Golich, associate professor of political science,was appointed as the first director of the center. Shortly thereafter she intro-duced CATs to Renee Curry, associate professor of literature and writing, andRegina Eisenbach, assistant professor of management. They were intrigued,and agreed to participate in a pilot to explore and document the value of CATsin a variety of disciplinary and course level settings—which we were wellplaced to do, as each of us taught a different student population: majors, gen-eral education students, and graduate students, respectively.

We selected CATs that we thought would be most adaptable to each of ourcontent areas and course objectives. We decided early in the process to incor-porate this new teaching strategy into those we already used successfully. Weoperated according to the belief that adopting new pedagogies only makessense if value is added to the students’ learning process. Each of us alreadyused, and continues to use, instruments designed to measure student learn-ing, such as quizzes, tests, papers, and oral examinations. By using CATs, wehoped to assess learning for development purposes rather than for gradingpurposes.

After choosing and adapting particular CATs—a mid-semester evaluation,pre and post confidence surveys, one minute papers, and muddiest point—weagreed on times during the semester to employ each but also allowed individ-ual variations. We wanted to collect evidence that would be comparable, butwe also wanted to test the flexibility and adaptability of the techniques. Wemet three times throughout the semester to discuss CAT results and to shareideas about the CAT procedure.

Description of CATs Used

The first CAT we adapted was the course-related self-confidence survey(Angelo and Cross, 1993); we called it the “pre/post” self-confidence survey. Thistechnique measures student content knowledge at the beginning and end of acourse. Each of us chose ten terms or ideas important in our individualcourses. We asked students, “How confident do you feel that you could explainthe following concepts to your friends or your parents?” Students completedthe “Pre” survey and kept it until the end of the semester when we had themcomplete the “Post” survey. Both surveys had the same ten terms, allowing stu-dents to compare the two to see how much they had learned.

At mid-semester, we created our own teacher-designed feedback forms (Angeloand Cross, 1993) to evaluate student perceptions of our course reading materi-als and format. There were scaled items as well as two open-ended questions.

Throughout the semester, students answered variations on the minutepaper (Angelo and Cross, 1993). One version asked students, “What informa-tion that we have covered today could have practical application for you out-side the classroom?” The other was “What was the muddiest point in today’sclass session?” Students answered these questions briefly at the beginning orend of a class period.


Interpreting the Feedback

The three CATs yielded valuable information regarding student learning andgeneral perceptions about our courses. To make the best use of this data, weneeded to share the results with the class by either a summary or a discussion.This completed feedback loop showed students that we took their responsesseriously; we spent valuable class time discussing them and frequently madechanges to the course based on this input.

Mid-Semester Feedback Evaluation. For all three of us, the quantita-tive section of this questionnaire pointed out areas of strength and areas forimprovement. What was additionally surprising was how similar our resultswere despite the differences in the contexts in which we taught. Most studentsin all three courses were satisfied with the reading materials, format, andknowledge the course provided. The open-ended comments led each of us tomake important modifications at the midpoint of the semester based on thestudent input. In the sections that follow each of us will speak in turn from thefirst-person perspective.

Eisenbach’s Management Class. In my management course, studentsbelieved that group activities, the class atmosphere, and focus on learninginstead of memorizing were the best things about the class. The negative com-ments concerned the exam format—“too much material, too little time”—andthe amount of outside time spent on the group project. In the large group dis-cussion, I explained how the interrelationship among the chapters requiredthat they be assigned together. And I designed the exam questions to integratethe knowledge in these related areas. Regarding the group project, the infor-mation from this CAT provided me the opportunity to reiterate the importanceof working with groups outside of regular class time.

Golich’s Political Science Class. In my political science course, the open-ended comments revealed that the students appreciated the real-world exam-ples found in the case studies used. Typically, negative comments focused onthe amount of reading required and the “dry” nature of the textbook; othersexpressed concerns about the emphasis on political economy since most polit-ical science courses are less interdisciplinary in approach. Perhaps the mostcritical outcomes of this evaluation derived from two different student com-ments that led to important “teachable moments.” First, a student wrote thatshe did not like “how I feel ‘duh’ after we discuss a case study.” When I reportedthe results of the overall evaluation, I commented that I wished I knew moreconcretely what that comment meant, since my goal was to help folks feelsmarter, not dumber! This brave student identified herself and explained thatshe thought she should be able to solve the complex issues in the cases alone;that her individual knowledge should be comprehensive enough to do this onher own. I was able to explain that one person rarely solves the real problems“out there”; rather, teamwork is imperative for identifying and implementingsolutions. This had a tremendous empowering effect for the students, who hadbeen trained to believe they should be able to do it all on their own.


The second instance came from a student who commented that I shouldchange “everything,” and that there was literally “nothing” to like about thecourse. During the feedback discussion, it became clear that this student wasan outlier. I was able to explain that I could not and would not change “every-thing,” and that, what is more important, if a class really needs an overhaul,comments must be more specific to help a professor implement the desiredchanges. These opportunities to clarify course goals and activities appear toengage students powerfully in their learning processes.

Curry’s Literature and Writing Class. Finally, in my literature and writingcourse, the open-ended comments demonstrated that the students liked theclass discussions and the variety of approaches to teaching. Most students per-ceived the course as useful and effective in attaining the goals of the course,which include reading, analyzing, and theorizing about the texts written byToni Morrison as well as those written about her work.

Interestingly, 47 percent of the students suggested that we spend moretime on the texts written by Morrison and less on the secondary texts. Thisresponse afforded me a wonderful opportunity to discuss with these first-semester master’s students what it meant to be graduate students in literatureand writing. The students and I discussed the importance of literary criticismand theory to the work of scholars and creative writers. Without this CAT, Iwould have assumed that graduate students inherently knew that the treat-ment of secondary texts increased as they moved into the master’s world.

Pre/Post Self-Confidence Survey. The pre/post survey results demon-strated a dramatic shift in student confidence. The pre surveys show the vastmajority of students not feeling confident about their knowledge of the tenconcepts. The post surveys show the exact opposite; students in all threecourses reported a much higher level of confidence at the end of the semester.

Eisenbach’s Management Class. In my management class, the results of thepost survey impressed the students. Comments such as “this proves we learnedsomething” were common. There was, however, one term that students didnot understand. I was initially concerned. The term, “Hofstede Framework,”should have been familiar to the students since it was recently covered in class.In the feedback session, I asked about the framework without referring to its“formal name” and the students were able to answer correctly. Had they notunderstood the underlying framework, students would not have been able toanswer the questions. Thus, this CAT helped me clarify the importance ofknowing the appropriate terminology that corresponds to theoretical models.

Golich’s Political Science Class. Likewise, in my political science class, stu-dents were extremely happy with the results of the pre/post self-confidencesurveys. Typical comments included the following: “I really like the pre andpost test. Although the pre test was embarrassing, I was happy to fill out thepost.” “This CAT makes you more aware of these terms as you come acrossthem in your readings.” And, “I liked the pre and post surveys because it gavethe course structure and showed what I learned over the semester.” In the oneinstance where students did not feel significantly more confident, I knew why!


I included the term “Postmodernism” in the surveys even though I knew wewould not cover it very explicitly in the course. When students registeredfewer “very confident” results for this term, I explained the importance of bothclass time and external reading to the learning process. That students felt lessconfident about this term indicated that we had not dealt with this concepteffectively in class.

Curry’s Literature and Writing Class. Again in my literature and writingcourse, the students enjoyed reviewing their post self-confidence surveys. Sim-ilar to an occurrence in political science, one student reported a “decrease” inself-confidence regarding the “Black woman writer” concept. This studentwrote on the form “Now I do not think I know ‘wimmin’ at all.” Upon dis-cussing this issue in class, we determined that phrases or labels initially appear-ing self-explanatory can become more complex, more political, and morecontroversial after further analysis. Therefore, self-confidence with a conceptmay decrease, indicating a more humble position toward the material ratherthan a failure to understand it.

Minute Paper and Muddiest Point. We all used the minute paper andmuddiest point CAT at least twice during the semester. This CAT proved mostbeneficial in helping us to modify our style and pace of delivery. It providedparticularly powerful “teachable moments” by allowing us to witness the exactintellectual struggle that was occurring for the students, and then to provokestudents in their thinking processes. The combination of anonymity and fac-ulty interest in student learning outcomes inspired honest, articulate responsesfrom the students.

In the end, students commented that the minute paper proved incrediblyvaluable to their individual and collective learning processes. In a separate eval-uation at the end of the semester in the political science course, they offeredthe following sample comments: “The minute paper allowed the class to par-ticipate in the lecture and allowed you to clarify tough questions.” “It reallyhelped to clarify the reading.” “It helped you [Dr. Golich] to see what we stilldid not understand.” “I felt as though I learned the material, rather than sim-ply being familiar with it.” “When you feel you have a ‘dumb’ question, thishelps.” “This one tests to see if students are ready for class, encourages studentsto become better prepared.” Perhaps most important, “The anticipation ofdoing this helped me keep up with the readings.”

Sharing the Results. In addition to sharing the information with stu-dents, we met once a month to show each other the results of each CAT. Thismeeting provided us the opportunity to process the information at a deeperlevel. The questions and comments we made to each other led us to reexam-ine the rationale behind our teaching methods. For the first time, we madeexplicit the assumptions underlying the way we teach our courses. Moreover,our discussions led us to test the validity of these assumptions. Thus our shar-ing sessions became opportunities for personal growth and development.

An added benefit of our regular meetings was the emotional support wereceived from each other. Sharing negative information about our teaching


placed each of us in a vulnerable position. However, the positive atmosphereof our group turned feelings of anxiety into feelings of confidence.

For example, one of us used the muddiest point to check whether studentshad done the reading. Unfortunately, most students had not prepared for thatday’s class. Using e-mail to explain this upsetting result allowed the rest of usto recommend courses of action and offer our understanding. This supporthelped defuse the potential anger and instead allowed an in-class discussion ofwhat it means to be prepared for class. Again, this reinforced the students’responsibility for their own learning. They came to realize that not doing thereading meant they missed out on important information. Thus the instant feed-back from colleagues allowed a “teachable moment” to occur in that class.

Conclusion

CATs provide professors with significant information about the value of coursecontent and the sometimes mysterious process of student learning. Creatingand using CATs reminded us that the most effective teaching begins with cleargoals for the entire course and for each individual class session. CATs con-tribute to self-reflection by both professors and students. Designing CATs helpsfaculty stay focused on course and class goals; responding to CATs requiresstudents to concentrate on course material and to think about what they arelearning. Thus, CATs demand a reciprocal effort from teacher and student tostay involved in the learning process throughout the semester.

It takes courage and self-confidence to work with CATs because we mustface and respond to occasional negative response from students. Therefore, fac-ulty have to feel ready to use CATs. Readiness often suggests we feel the needto take a reality check on what is or is not working in our classes. It impliesthat we consider change may be necessary and desirable. At the same time,CATs provide opportunities for students to articulate why they may not wantto make certain changes. By forcing us to return to our learning goals repeat-edly, CATs encourage a self-reflection which can let us know what changeneeds consideration, when such a change warrants consideration, and whyarticulation of learning objectives needs repetition.

This chapter describes a cross-sectional study that compares CAT val-ues across two dimensions, disciplines and course levels. We havedescribed how the use of CATs fosters improved communication and reflec-tion. We invite readers to try this themselves; the experience will be bothrefreshing and rewarding.

References

Angelo, T. A., and Cross, K. P. Classroom Assessment Techniques: A Handbook for CollegeTeachers. (2nd ed.) San Francisco: Jossey-Bass, 1993.

Association of American Colleges. Integrity in the College Curriculum: A Report to the Aca-demic Community. Washington, D.C.: Association of American Colleges, 1985.

Boyer, E. L. Scholarship Reconsidered: Priorities of the Professoriate. Princeton, N.J.: Carnegie


Foundation for the Advancement of Teaching, 1990.Chickering, A. W., and Gamson, Z. F. “Seven Principles for Good Practice in Undergradu-

ate Education.” AAHE Bulletin, 1987, 39, 3–7.Cohen, D. K. “Teaching Practice: Plus ça Change. . . .” In P. W. Jackson (ed.), Contributing to

Educational Change: Perspectives on Research and Practice. Berkeley, Calif.: McCutchan,1989.

Dewey, J. How We Think. Lexington, Mass.: Heath, 1933.Ehrmann, S. C. “Local Assessment of Educational Strategies That Use Computing, Video,

and Telecommunications.” Commissioned paper prepared for AAHE’s 10th Annual Con-ference on Assessment and Quality, Boston, Mass., June 11–14, 1995.

Ewell, P. T., and Jones, D. P. “Actions Matter: The Case for Indirect Measures in AssessingHigher Education’s Progress on the National Education Goals.” Journal of General Educa-tion, 1991, 42 (2), 123–148.

McKeachie, W. J., Pintrich, P. R., Lin, Y. G., and Smith, D. Teaching and Learning in the Col-lege Classroom: A Review of the Research Literature. Ann Arbor, Mich.: National Center forResearch to Improve Postsecondary Teaching and Learning (NCRIPTAL), University ofMichigan, 1986.

Pascarella, F. T., and Terenzini, P. T. How College Affects Students: Findings and Insight FromTwenty Years of Research. San Francisco: Jossey-Bass, 1991.

Stice, J. E. “Using Kolb’s Learning Cycle to Improve Student Learning.” Engineering Educa-tion, 1987, 77, 291–296.

REGINA EISENBACH is associate professor and center director, Center for Service SectorManagement in the College of Business at California State University at San Marcos.

VICKI L. GOLICH is professor of political science and director of the Faculty Centerat California State University at San Marcos.

RENEE CURRY is associate professor of literature and writing at California State Uni-versity at San Marcos.

Documents

Classroom Assessment Across the Disciplines