Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity

Almost every academician has heard the term “scholarship of teaching.” However, exchanges among colleagues from many institutions reveal a heterogeneity of meanings, attitudes, and experience levels with this topic. A comprehensive and coherent framework seems needed to accommodate the diverse strands of thought and endeavor. Similarly, a treatment of SOTL-related topics that accommodates a broad continuum of familiarity and experience seems called for.

The tutorial attempts to meet those needs, by providing a “primer” … a useful grounding in the whole domain of SOTL-related topics.

Materials for the tutorial originated in workshops and conferences sponsored by the American Association of Higher Education (AAHE), the Carnegie Foundation for the Advancement of Teaching, the Professional and Organizational Development Network in Higher Education (POD), the Lilly Conferences on College

Teaching and various individual colleges and universities.

Your hosts: Samuel B. Thompson, Craig E. Nelson, and Rita C. Naremore

A group making the journey together along the continuum “from Alpha to Omega” will find interesting topics for discussion and rich resources for further study. The tutorial is well enough indexed to also serve encyclopedically for those who seek information about specific SOTL-related topics.

“The tutorial speaks to so many of the interests and concerns of faculty members and administrators who are grappling with issues of teaching, scholarly teaching, and the scholarship of teaching and learning. You and your colleagues have done a tremendous job in organizing and presenting these materials.” Barbara Cambridge

Welcome to a Tutorial on the Scholarship of Teaching and Learning (SOTL)

Using the Tutorial Click at any point on the screen to advance to the next frame. Move your mouse to show faint arrows at the lower left corner of the frame. Click on either of these arrows to show a menu. To move backward one frame, click on (Previous).

Jumps to frames elsewhere in the tutorial are dark blue and underlined. To return to the frame from which you jumped, use (Go, then Previously Viewed). This two-step procedure will probably be frequently used. Click on (Go then Slide Navigator) to see a menu of frames and to move to any point in the tutorial.

To end the tutorial, use (End Show)

Other types of embedded resources linked to the main path are indicated by icons:

Website Video

Audio

Respond to a task (click on icon)

This is a multimedia tutorial including photos, graphics, hyperlinks to sources, audio and video clips, and interactive tasks as well as text.

The tutorial is organized in frames (not pages) consisting of short blocks of information about the topic in the frame title. The continuum is traversed by proceeding serially along a main path through the frames, stopping to examine embedded resources whenever they seem of interest. There are many frames, and it is not intended that the viewer intent on mastery of content will travel through all of them in one, or even two sittings. However, we have tried to make the journey entertaining as well as informative. Travel through the frames goes quickly, information is conveyed in various ways, and many good stopping places are evident.

“It’s really impressive to see how much you have compressed and organized into a manageable ‘package.’ I confess I was a bit daunted by the number of frames at first, but the tutorial moves along very smoothly. A very efficient way to learn this stuff.” Pat Hutchings

Acknowledgements

Moya Andrews

Thomas Angelo

William Becker

Simon Brassell

Barbara Cambridge

William Cerbin

Donetta Cothran

Patricia Cranton

K. Patrica Cross

Alix Darden

Robert Diamond

Donna Duffy

David Pace

Edward Redish

Eugene Rice

Laurie Richlin

Leah Savion

Anita Salem

Donald Schon

Nancy Shanklin

Lee Shulman

John Sullivan

George Walker

Barbara Walvoord

Individuals Institutions

Abilene Christian University

The Citadel

Elon College

Indiana University

Middlesex Community College

Notre Dame University

Rockhurst University

Information and materials from more than 30 individuals across the United States and Canada as well as SOTL program information from seven colleges and universities are used in this tutorial. All contributions are gratefully acknowledged.

Charles Glassick

Samuel Guskin

Mary Huber

Pat Hutchings

Robert Hutchins

Dennis Jacobs

T. Mills Kelly

Carolin Kreber

Marcia Landen

Suzanne Mabrouk

Gene Maeroff

Valerie O’Loughlin

This project is sponsored in part by an American Association of Higher Education (AAHE) Grant

© 2001 Samuel B. Thompson

1. How would you describe your experience with scholarship of teaching and learning?

Choose one: Extensive Moderate Hardly-Any None

2. OUTCOMES: List 3 topics or issues you hope to learn more about from this tutorial

STOP NOW! BEFORE YOU GO FURTHER, WE NEED YOUR RESPONSES TO THIS FRAME. This task is DESIGNED to capture experience level of users and their expectations/goals in freeform before influence by tutorial content. If you will be so kind as to activate the button in the slide and type your responses, we’ll put the information you provide to good use!

Task A

Boyer coined the term “scholarship of teaching” more than ten years ago. The American Association of Higher Education (AAHE) and the Carnegie Foundation for the Advancement of Teaching have conducted initiatives to advance this scholarship for several years. Now, a continuum of experience levels is evident among academicians. At the “Alpha” end of the continuum, people have heard the term but are unsure what it means. These need perhaps a glossary and basic grounding in the key developments over the last decade. In the middle are people who may have launched campus initiatives or may be wondering whether particular issues and projects could result in scholarly products. At the “Omega” end, those at the “cutting edge” of SOTL endeavor to construct and validate developmental processes as sustainable bridges to carry faculty from initial interest to stages of productivity that achieve goals and count in reward systems.

This tutorial addresses the whole continuum of SOTL experience from “Alpha to Omega.” The three sections of the continuum correspond to the three units of this tutorial and each phrase names a module … so two modules per unit, six total. The first unit, features essential definitions, concepts, rationale and other background. The second unit features administrative “how to” and examples of campus programs as well as faculty projects. The third unit provides some research essentials to permit viewers to launch their own projects. The “metaperspective” afforded in considering the whole continuum permits viewers to structure their own projects and campus initiatives.

Αlpha Omega

1A What & Why of SOTL

1B Origins of SOTL

2A Initiating SOTL Programs

2B Faculty SOTL Projects

3A Bridges to Productivity

3B Questions, Designs & Resources

Continuum of Experience with SOTLA Metaperspective

Alpha to Omega Continuum:Contents

Alpha Omega

Unit 1A

•“The “What” of SOTL

•Why SOTL?

•Task B: Reasons to Engage

Unit 1B

•Overview

•Reform Concepts

•Implementing Entities

•Recent Articulations

•Task C: Critique of Origins

and Evolution

Unit 2A

•A Campus Example in Detail

•Task D: Refining Your Campus Program

•Examples on Other Campuses

•Task E: Applicable Features

Unit 2B

•Donetta Cothran – Kinesiology

•Valerie O’Loughlin – Medical Sciences

•David Pace – History

•William Becker - Economics

•Rita Naremore – Speech and Hearing Sciences

•Dennis Jacobs – Chemistry

•Leah Savion – Philosophy

•Other Examples – Carnegie Scholars

•Task F: Minute Paper

Unit 3A

•How Could I Do SOTL (Genres)

•Task G: Reflecting on Genres

•Approaches to Scholarship Via Classroom Research Projects

•Where to Publish and Present

•Sources of External Funding

•Task H: Classroom Research

Unit 3B

•Framing Questions

•Task J: Examining Valerie’s Questions

•Task K: Framing Your Question

•Designs for Studies

•Choosing Measures

•Guiding Questions in Choosing Methodology

•Task L: Designing Your Project

•Summary of Standards

Unit 1AThe What and Why of SOTL

We seek understanding of what SOTL is and its relationship to teaching as well as reasons for becoming engaged in SOTL.

•“The What”: A Conceptualization of Teaching-Related Activities

•Why SOTL?

•Task B: Your reasons to become engaged in SOTL

Scholarly Teaching: Teaching that entails certain practices of classroom assessment and evidence gathering; teaching that is informed not only by the latest ideas in the field but by current ideas about teaching generally and specifically in the field; and teaching that invites peer collaboration or review.

Scholarship of Teaching:An act of intelligence or artistic creation becomes scholarship when it possesses at least three attributes: it becomes public, it becomes an object of critical review and evaluation by members of one’s community, and members of one’s community begin to use, build upon, and develop those acts of mind and creation.

(Definitions after Hutchings and Shulman)

Teaching ScholarlyTeaching

Scholarship of Teaching &

Learning

Many conceptualizations of SOTL have been offered over the years. This one is ours. Others will be exhibited in the next module.

Every teaching-related act can be represented as a point somewhere within the three elliptical regions. Elliptical shapes are chosen solely for convenience of representation. All deeds that teachers perform in the name of carrying out their teaching assignments can be imagined as individual points within the large ellipse labeled “teaching.” The centillions of such points include acts of materials development, grading, individual tutoring … every teaching activity in or out of the classroom. Within this big red ellipse of teaching acts, we can identify a subset that meet special criteria for “scholarly teaching.” The smallest ellipse (yellow) represents “scholarship of teaching and learning.” Scholarship of teaching and learning (SOTL) is rooted in teaching but extends beyond it (out of the red ellipse). SOTL involves creation and dissemination of original work that makes a useful contribution to knowledge and practice of other teachers.

Though sharply defined in the picture, the boundaries of ellipses can seem a bit fuzzy when classifying certain teaching-related acts. This occasional fuzziness does not lessen the usefulness of this seemingly simplistic conceptualization.

More about Lee Shulman More about Pat Hutchings

A Conceptualization of Teaching Related Activities

An excellent deed of teaching

A superior deed of scholarly teaching

An ineffective deed of teaching

Quality (or “excellence”) of any teaching-related activity is an independent dimension not represented in the plane of the Venn diagram.

Faculty members should be encouraged to move acts of teaching that are characterized by points in the red toward the orange … in other words toward scholarly teaching. To the extent that teachers adopt practices of scholarly teaching, their teaching will probably improve. Similarly, their acts of scholarship will probably improve their teaching, develop them professionally, and add to our body of useful knowledge about practice. However, we must stop short of accepting either the proposition that scholarly teaching implies excellent teaching or the converse. Similarly scholarship of teaching does not imply excellent teaching.

“Excellence” is often confounded with “scholarly teaching” and “scholarship of teaching.” Scholarly teaching practices do not necessarily confer excellence. All seasoned faculty know of colleagues who are excellent teachers … excellent in terms of achieving superb outcomes in students … without their meeting any of the criteria of “scholarly teaching” Avoidable indignation and opposition to SOTL initiatives are justifiably engendered in faculty by the notion that “scholarly teaching” and “scholarship of teaching” are necessarily what one must do to be considered excellent in teaching.

No ascending hierarchy of excellence can necessarily be associated with movement from left to right or bottom to top in the Venn diagram. To represent excellence, we must invoke a third dimension perpendicular to the plane of the diagram. In this three-dimensional conception, each teaching-related deed represented by a point in the diagram has associated with it an arrow (a “quality vector”) representing excellence (or lack of it). The direction of the arrow indicates whether the deed is above or below average in quality and the length of the arrow indicates how far above or below average.

Relationship to Excellence

Distinction between scholarship of teaching and learning as a domain of academic activity and as a campus program: • SOTL as academic activity (yellow ellipse)• SOTL as a campus program can span all teaching

- related activities (leftmost three ellipses).

A SOTL program provides support to faculty for movement toward the yellow ellipse; that is movement in their teaching-related activities toward the right and movement in their research-related activities toward the left.

Teaching Scholarlyteaching

Research and creative activity

Scholarship of teaching and

learning

Into what realm beyond teaching does scholarship of teaching extend? The rightmost ellipse is the set of points representing deeds that scholars perform in the name of research and creative activity. Though research-based activity to advance knowledge is the prevailing notion of scholarship, George Walker tells us that scholarship is not precisely synonymous with research. Scholarship is reflective and creative. It can result from reflecting on the processes and outcomes of research or personal teaching experience and creating something new.

Boyer’s notion that scholarship of teaching and learning spans the “tired teaching vs. research debate” is naturally depicted in the position of the yellow ellipse (with italicized text) between the two big red ellipses.

We can distinguish between SOTL as an academic activity and as a campus initiative. As an academic activity, what scholarship of teaching and learning means in your institution defines the interior and boundary of the yellow ellipse. As a campus initiative, what SOTL means may encompass the entire 3 leftmost ellipses. In other words, a SOTL initiative may include all teaching related activities of faculty. Thusly conceived, a SOTL initiative can provide a resource-rich and opportunity-laden home to faculty members who aspire SIMPLY to teach excellently as well as to those who perceive themselves primarily as scholars (i.e.”researchers”). In fact, there is evidence that faculty who have not been attracted to previous teaching improvement initiatives can be attracted to a SOTL initiative because of its research orientation. This phenomenon may be visualized in the diagram as faculty converging to SOTL not only from the big red ellipse on the left but also from the one on the right.

SOTL as Academic Activity vs. Campus Initiative

• Unite Where Teaching and Pursuit of New Understanding are Shared Interests

• Build Multidisciplinary Community

• Capture the Wisdom of Practice

• Foster Mentoring of New Generations of Teachers

• Employ Reserve Intellectual Capacity

• Recreate the Concept of Scholarly Excellence

Elaboration of the unifying potential of SOTL (Voice of Samuel Thompson)

Elaboration of community-building via SOTL (Voice of Samuel Thompson)

Documentation of the deep and useful insights gained over time so that others can use them is encouraged by SOTL. The documentation includes the teaching moves and materials that every seasoned faculty member has found to work especially well.

SOTL is a medium for the generativity that develops naturally in mid-life … seasoned teachers helping newer ones to learn to teach more effectively.

SOTL may increase the individual and collective scholarly productivity of the faculty, partly by providing an avenue to products from the immense investments that already go into teaching and partly by providing an area where faculty productivity can thrive when scholarly efforts in other areas are flagging or no longer interesting.

SOTL brings the spirit and standards of scholarly excellence to teaching .. standards that have traditionally been applied to other fields of scholarly endeavor but not to teaching.

SOTL can improve teaching and learning … and more! SOTL has power to:

Why SOTL?

Why SOTL? (Cont.)

Knefelkamp L. L. (1990) Seasons of Academic Life: Honoring Our Collective Autobiography Liberal Education 76. 3

• Power of Ideas

• Faculty Role

• Students

• Public Self

• Multiple and Competing Commitments

• Need to Step Out

• Marginality

• Courage

SOTL Honors and Enriches the Seasons of Academic Life Elaboration of the role of SOTL in the seasons of academic life (Voice of Craig Nelson)

Knefelkamp’s characterization of Seasons of Academic Life seems worthwhile reading for any faculty member. This particular article may serve as a basis for a meaningful exercise in deepening understanding of “Why SOTL?” The recommended exercise is to read Knefelkamp’s description of each season and then pause to reflect on the possible roles that scholarship of teaching and learning may take during it and the implications for the faculty member.

• What reasons to become engaged in SOTL … the ones discussed in the tutorial or others … are most important on your campus?

• What reasons might be irrelevant to faculty on your campus?

We pause here for processing of the foregoing and for reflection. Please activate the button and e-mail your reasons. We’ll compile categories of reasons and disseminate the results.

Task B: Reasons to Engage in SOTL

mailto:

Unit 1B Origins and Evolution of SOTL

•Overview

•Reform Concepts

•Implementing Entities

•Recent Articulations

•Task C: Critique of Origins and Evolution

As we progress to the second and subsequent modules, we’ll continue to broaden and deepen our concept of “what SOTL is” and “why become engaged in SOTL.”

We invite your peer review and critique of this module. At the end of the module, we will ask you (Task C) to tell us how you would modify the framework and which elements you would add or omit in each the first four categories above. We’ll consider your inputs and may incorporate them in future versions of this tutorial in order to build a better framework for everyone.

Not a new idea (Hutchins 1923)

“Scholarship of teaching” coined (Boyer 1990)

Reform Concepts

Classroom Assessment/ Research

New Epistemology

New American Scholar

Conception of Teaching

Scholarship Assessed

Implementing Entities

AAHE

Carnegie

Pew

Lilly

Recent Articulations

Scholarship of Teaching

Scholarly Teaching

Model of SOTL

Relation of ST to SOTL

Origin and Evolution of SOTL (In Progress)

The “work in progress” in this “tree” is our tentative attempt to build a succinct framework for essential background in SOTL. Of course, the number of individuals, forums, articles, and books that have significantly contributed to SOTL is too enormous to include comprehensively. And who is to say which of these is “essential”? So this whole exercise is somewhat arbitrary and no doubt reflects the bias of our own limited knowledge and experience.

“What is certain is that most Ph.D.’s become teachers and not productive scholars as well. [A Ph.D. candidate who plans to be a teacher] must know his field and its relation to the whole body of knowledge. It means too that he must be in touch with the most recent and most successful movements in undergraduate education, of which he now learns officially little or nothing. How should he learn about them? Not in my opinion by doing practice teaching upon the helpless undergraduate. Rather he should learn about them through seeing experiments carried on in undergraduate work by the members of the department in which he is studying for the degree….”

Robert Maynard Hutchins

The quote is taken from Hutchin’s inaugural address as the fifth president of the University of Chicago in 1928 and is presented as evidence that the concept of SOTL existed long before Boyer popularized the term. The last sentence suggests that faculty in all departments should carry on experiments in undergraduate teaching and learning … a fundamental activity of SOTL. The sentence further suggests that graduate students preparing for academic careers should learn from these experiments.

Note that Hutchins also presaged the plight of many undergraduates, and the need for programs to prepare graduate students to teach … programs like the Preparing Future Faculty (PFF) initiative of the 1990’s.

In the first sentence, Hutchins also expressed a truth still not internalized by many faculty who prepare graduate students in doctoral institutions today … research productivity that advances the discipline ends with the dissertation for most Ph.D’s. Today, the vast majority (95% is a figure often quoted) of new Ph.D’s who take positions in academe do so in teaching institutions rather than research universities. In other words, though they may acquire their Ph.D’s in research universities as researchers, teaching is what they will primarily do in academic careers.



Reform Concepts


New epistemology

New American scholar




AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL

Scholarly Teaching and Scholarship

Not a New Idea

http://www.preparing-faculty.org/

“Scholarship of Teaching” CoinedNot a new idea (Hutchins 1923)


Reform Concepts


New epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Ernest Boyer popularized “scholarship of teaching” in a famous monograph Scholarship Reconsidered: Priorities of the Professoriate. 1990 Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.

Boyer argued that the concept of scholarship must be broadened to include not only basic research but other kinds of intellectual work in which faculty engage. He suggested four types of scholarship: discovery, integration, application, and teaching. All of the elements of “Reform Concepts” build on Boyer’s theme of broadening the traditional view of scholarship.

Boyer disseminated the notion of a scholarship of teaching but did not define clearly what this scholarship would be. He thereby touched off a decade of academic thought and controversy over this topic. As a result, our conceptualization of this scholarship is a bit clearer today (but only a bit!)

Reform Concepts:Classroom Assessment/

Classroom Research



Reform Concepts


New epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Reform ConceptsThe several reform concepts presented in this section of the tutorial may be viewed as multiple origins or histories of SOTL. They are somewhat overlapping and sometimes difficult to isolate.

Classroom Assessment Techniques (CATs)K. Patricia Cross and Thomas Angelo pioneered this “do-it-yourself” approach to assessing conditions of learning in classrooms. Their 1993 book, Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) Jossey-Bass is a famous work of scholarship of teaching and learning, reported to have sold more than 35,000 copies. Thousands of college and university teachers use CATs in their classrooms today, especially versions of the “one-minute paper,” and there is evidence of improved learning as a result. Angelo and Cross contributed to the current focus on learning but also hoped their book would influence faculty to go beyond application of CATs in the classroom … that it would in fact spark systematic classroom research which would in turn become a popular form of SOTL nationally … a form more useful to faculty than traditional educational research. The adoption of CATs became widespread, the adoption of classroom research much less so.

Classroom Research In 1996, Cross with Mimi Steadman published a sequel book, Classroom Research: Implementing the Scholarship of Teaching, Jossey-Bass placing CATs in a broader research context. Again the hope was to institutionalize classroom research. However, as of yet, classroom research is still not nearly as widely employed as are CATs.

More on CATS More on Classroom Research

Reform Concepts:New Epistemology



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


“On the high ground, manageable problems lend themselves to solution through the use of research-based theory and technique. In the swampy lowlands, problems are messy and confusing and incapable of technical solution. ...the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. ...Shall [the practitioner] remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?”

Donald Schon

Donald Schon posed a “dilemma of rigor or relevance” for research epistemology. In this eloquent quote, he calls for new research epistemology for the scholarship of teaching and learning based on reflection by teachers on actions taken in their practice.

Schon, D. “The New Scholarship Requires a New Epistemology,” Change, Nov./Dec. 1995 27 ( 6) p. 28

Reform Concepts: The New American Scholar



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


CONCRETE CONNECTED

KNOWING

ABSTRACT ANALYTIC KNOWING

ACTIVE PRACTICE REFLECTIVE

OBSERVATION

R. Eugene Rice. 1996. Making a Place for the New American Scholar. AAHE. p. 14

Adapting the learning cycle of David Kolb to SOTL, Gene Rice sees traditional scholarship … particularly the so-called “social science model” … as characterized by reflective observation and abstract analytic knowing. It therefore resides in the lower right quadrant of his diagram. He asserts that SOTL needs to be rooted more in active practice and concrete connected knowing. Said another way, scholarship of teaching and learning should take forms that properly reside in the upper left quadrant

Reform Concepts:Conception of Teaching

&Scholarship Assessed



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Conception of Teaching: The argument here … and Lee Shulman is a major proponent … .is that teaching is not just technique (though technique has gotten the lion’s share of attention in teaching-improvement efforts) but an enactment, rather, of our understanding of our disciplinary, interdisciplinary or professional field and what it means to know it deeply. As Shulman wrote recently "a scholarship of teaching will entail a public account of some or all of the full act of teaching—vision, design, enactment, outcomes, and analysis—in a manner susceptible to critical review by the teacher’s professional peers, and amenable to productive employment in future work by members of that same community."

Scholarship Assessed: Glassick, Huber, and Maeroff (1997) in Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass review the history of scholarship in American academe and describe general standards for all forms of scholarship. These standards “pull together” many of the concepts noted in this section. More detail on the standards

Summary of Reform Concepts. All reform concepts contain an implicit call to broaden and generalize traditional notions of scholarship in order to make progress against relevant problems which vex undergraduate teaching and learning at levels of individual teacher and course. George Walker advises us to avoid unnecessary limitations in our notions of scholarship. The goal is to produce a body of scholarship that may deviate from conventional forms, and that may not be applicable over disciplines, categories of student, or institutional type but which nevertheless is more used by teachers and contributes more to the learning of students than does past educational research.

Implementing Entities:AAHE



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


AAHE (American Association of Higher Education) The AAHE has sponsored many initiatives over the years to advance teaching and its scholarship. Barbara Cambridge directs the Teaching Initiatives. Major initiatives include assessment, service-learning, peer-review of teaching and teaching-learning technologies. We highlight faculty “roles and rewards” below because its historic origin is associated with Boyer’s introduction of “scholarship of teaching.”

AAHE FFRR (Faculty Forum Roles and Rewards) It was recognized at the time of Boyer’s 1990 monograph, that overhaul of the faculty reward system needed to go hand-in-hand with advancement of scholarship of teaching, if the latter was to be more than a transient topic. The FFRR has met annually since 1990 to address faculty roles and rewards.

Robert Diamond’s depiction of the factors bearing on faculty work reflects the agenda of AAHE FFRR. Initiatives have been undertaken in the 1990s’ to favorably influence each of the five factors in the figure toward greater incorporation of SOTL in faculty work. Factors Influencing What Faculty Do

Implementing Entities We focus on broad national initiatives. There are far more entities (including many academic institutions) deserving acknowledgement for implementation than those included here.

Diamond, R.& Adams, B. The Disciplines Speak AAHE 1995 p.7

http://www.aahe.org/

Implementing Entities: Carnegie



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


THE CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING The Carnegie Foundation has a number of exemplary programs and resources that encourage scholarly teaching and the scholarship of teaching and learning. Among the programs that may be explored are: Cultures of Teaching and Learning in Higher Education, Higher Education and the Development of Moral and Civic Responsibility (co-led by Tom Ehrlich, former President of Indiana University), Preparation for the Professions, Teacher Education, and the long running U.S. Professors of the Year. Four key programs to click on:

Rethinking the Doctorate. George E. Walker, Vice President for Research and Dean of the University Graduate School at Indiana University, leads this new, five-year project.

Knowledge Media Laboratory. A gallery of course portfolios illustrating various approaches to the scholarship of teaching. Relation between course portfolios and SOTL

Resources. Click on “e-Library” to read key SOTL references on-line and to access an annotated bibliography of SOTL in higher education.

CASTL Carnegie Academy for the Scholarship of Teaching and Learning

http://www.carnegiefoundation.org/OurWork/index.htm

http://www.carnegiefoundation.org/AboutUs/Staff/walker.htm

• Pew National Fellowship Program for Carnegie Scholars For faculty from higher education - brings together outstanding teacher-scholars from a variety of academic fields and

institutions to pursue investigations of issues in teaching and learning.

• Teaching Academy Campus Program Coordinated for the Carnegie Foundation by AAHE. Institutions in higher education that are prepared to make a public

commitment to the scholarship of teaching and learning are invited to join the Campus Conversations program. As progress is made, campuses are invited to apply for small grants for the Going Public phase. CASTL now is also encouraging the formation of National Networks of Teaching Academy Campuses. The AAHE CASTL Campus Program Web Center includes a number of resources for thinking about SOTL, including reports from campus conversations, a list of going public grants and selected resources.

• Work with the Scholarly and Professional Societies Carnegie is encouraging societies to increase their emphasis on teaching and on the scholarship of teaching and

learning. This is a continuation of work under AAHE FFRR culminating in two publications which contain perspectives of disciplinary societies on Boyer’s four scholarships: The Disciplines Speak AAHE, 1994 and The Disciplines Speak II AAHE, 2001.



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)

Implementing Entities:Carnegie (cont.)

CASTL

http://aahe.ital.utexas.edu/

http://www.carnegiefoundation.org/CASTL/index.htm

Implementing Entities: PewPeer Review of Teaching

Course Portfolios



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Pew Charitable Trusts:

•CASTL (with Carnegie Foundation)

•Peer Review of Teaching

Peer Review of Teaching is an ongoing area of work intertwined with the scholarship of teaching and learning in several ways.

Peer Review has been an emphasis of AAHE since 1994 and continues to be supported by the Pew Foundation and other grant sources. The course portfolio has evolved as the principal means by which peer review of teaching is effected. Two notable current programs to advance course portfolios, both supported by Pew, are centered at the University of Nebraska Lincoln and Samford University

Individual course portfolio projects and SOTL projects are so closely related as to often seem inseparable. Example of this inseparability.

Perhaps the best way to ensure that quality teaching is recognized, valued, and rewarded is to improve the means of identifying and documenting teaching effectiveness. Course portfolios afford a comprehensive yet efficient means of documenting the intellectual work of teaching a particular course. Through such a portfolio, a faculty member documents course design and execution, including results in student learning. In this way, teaching can be understood and presented as a form of scholarship, utilizing the accountability through peer review that already exists in higher education. A course portfolio can be used as an instrument for exhibiting teaching effectiveness, a framework for cultivating scholarship, and a vessel for conveying one's work to appropriate publics, including promotion and tenure committees.

http://www.aahe.org/teaching/Peer_Review.htm

http://www.unl.edu/peerrev/

http://www.samford.edu/pbl/page3.pdf

http://www.samford.edu/pbl/page3.pdf

Implementing Entities: Lilly Conferences on

College Teaching



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Lilly Conferences are retreats that combine workshops, discussion sessions, and major addresses, with lots of opportunities for informal discussion about excellence in college and university teaching and learning. Internationally-known scholars join new and experienced faculty members and administrators from all over the world to discuss topics such as gender differences in learning, incorporating technology into teaching, encouraging critical thinking, using teaching and student portfolios, implementing group learning, and evaluating teaching. The welcoming "Lilly Spirit" and the high level of scholarly attention to teaching and learning enable everyone to contribute to the forum.

Lilly Conferences have encouraged scholarly work in teaching and provided a venue for faculty members to discuss and disseminate such work for more than 20 years.

http://www.muohio.edu/lillyconference/

Recent Articulations:Scholarship of Teaching

Scholarly Teaching



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


We have left the topic of “Implementing Entities”, and moved to “Recent Articulations” (the rightmost branch of the “origin and evolution tree”. ) A decade after Boyer’s “Scholarship Reconsidered”, the meaning and implications of scholarship of teaching and learning are topics of evolving thought and discussion. Many more entries than the four shown could be included in the right branch and these would be no less worthy than the sources shown. However, our purpose is to provide a sample rather than a comprehensive list. Our apologies to authors for failing to list their contributions.

Scholarship of Teaching and LearningThis recent articulation (Shulman,Change 31 (4) 1999) is widely accepted. Note that it is not quite a definition; it offers necessary conditions for scholarship but not sufficient conditions. The articulation does suggest a broadening of the concept of scholarship as advocated in the Reform Concepts Applications of the broadened concept of scholarship.

Scholarly TeachingThis articulation (Hutchings & Shulman Change 31 (5) 1999) is also widely accepted. One of it’s principal features is to include knowledge of recent and relevant developments in pedagogy as well as the traditional notion of recent and relevant developments in the field as a criterion for scholarly teaching.

Recent Articulations: Exploring Scholarship of

Teaching The Model of Kreber &

Cranton



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Teaching

Content Reflection

Process Reflection

Premise Reflection

Pedagogical Knowledge

Kreber, C. and Cranton, P. Journal of Higher Education. Vol 71, No. 4.

Carolin Kreber and Patricia Cranton see development of scholarship of teaching as a process of reflection on experience and research in teaching.

The second, third and fourth concentric circles represent content, process, and premise reflection respectively. The three spokes partition the circles into regions representing the three domains of knowledge (instructional, pedagogical, and curricular) shown on the periphery of the figure.

Within each knowledge domain, learning is characterized by various forms of instrumental, communicative, and emancipatory learning processes, which are derived from Habermas’s three forms of knowledge.

Development of SOTL in this conceptualization is thus comprised of nine distinct types of learning.

Recent Articulations: Conceptualizing Scholarly Teaching

and Scholarship of Teaching



Reform Concepts


New Epistemology





AAHE

Carnegie

Pew

Lilly



Scholarly Teaching

Model of SOTL


Laurie Richlin’s conceptualization reflects her extensive experience as co-director of the Lilly Conferences on College Teaching and in helping faculty with teaching-related investigations. Her conceptualization shown in the slide is closely tied to the ideas of classroom research. She partitions a developmental process for SOTL according to the aspects that can be considered scholarly teaching and scholarship of teaching. Her steps in the part of the process called scholarly teaching are similar to those in our classroom research schematic.

Richlin, L in Kreber, C. (Ed.) New Directions in Teaching and Learning 86 2001 (In Press)

Task C

Think about your own knowledge of SOTL. What additions would you make to the framework “Origins and Evolution of SOTL” in each category?

Reform Concepts



We pause to process, reflect on, and improve the “Origins and Evolution of SOTL.” WE NEED YOUR HELP WITH THIS! Please suggest alterations to the structural framework of the tree as well as to the specific elements in the branches. Just click on the button on the frame to e-mail your suggestions. We’ll be most grateful for your contributions, and they will result in a better product for everyone.

mailto:

Unit 2AInitiating SOTL Programs

•A Campus Example in Detail ( Indiana University Bloomington)

•Task D: Refining Your Campus Program

•Examples on Other Campuses

•Task E: Applicable Features

We are leaving the left region (Unit 1) of the Alpha to Omega Continuum and transitioning to the middle region (Unit 2).

• Identifying Key Resources• Getting People Involved• Changing the Institutional Culture• Assessing SOTL Program Impact• Task D Refining Your Campus Program

An alternative path through this unit is to view “Assessing SOTL Program Impact” first. After studying “A Campus Example in Detail”, you will be asked to reflect on whether any aspects of this example are applicable to your own campus (Task D).

A Campus Example in Detail:Using SOTL to Make Change Happen

Three Stages of Campus Program Goals

Participation

Scaffolding

Scholarly Productivity

Campus program development may be viewed in a framework of three stages. The first stage is to build awareness and participation. The second is to erect essential scaffolding to support faculty in this line of work. The third is to maximize scholarly productivity.

It may be helpful to bear the above three stages in mind while studying this detailed example of a campus program. To assess the extent to which goals of the first two stages are achieved, look for evidence distributed through the frames of this unit. Evidence of scholarly productivity will be largely found in the following unit “Faculty SOTL Projects.”

The Ingredients:– Massive administrative support

– A dedicated director

– A faculty advisory council

– A core of faculty members who are willing to engage in this work and share it with others

Moya Andrews

Dean of the Faculties and Vice Chancellor

for Academic Affairs

What are the Key Resources?

Massive administrative support

Moya Andrews is committed to the campus SOTL initiative and an architect of it from its inception. Her administrative support is massive and visible to all. She provides a budget for the SOTL initiative. However, she also contributes vitally by staying abreast of SOTL-related developments and directing the campus initiative. She makes public statements, writes messages to faculty and invitations to department heads and other key individuals. She often introduces featured presenters at events.

Message to faculty regarding SOTL

http://www.indiana.edu/~deanfac/sotl/eventsintro.html

Massive Administrative Support (Cont.)

George Walker Vice President for Research and

Dean of the Graduate School

“Expanding our Vision of Indiana University’s Research Mission: The Scholarship of Teaching and Learning”

Vice President Walker’s talk at the first SOTL main event was subsequently delivered to representatives of research universities at invitation of AAHE and distributed nationally.

Other SOTL resources by George Walker:

A statement that SOTL is important.

A brief message to faculty regarding the rationale and significance of SOTL.The campus SOTL initiative received a huge boost in administrative

support when George Walker became a partner in it. From that time, the SOTL initiative was impelled by the two primary academic offices on the campus. This partnership facilitated the movement of faculty interest toward SOTL from research as well as teaching. George Walker describes some of the administrative support available for SOTL.

George Walker is currently a senior scholar at the Carnegie Foundation leading a new study: Rethinking the Doctorate

George Walker’s contributions exemplify what a distinguished researcher and senior administrator can do to help advance teaching and its scholarship.

http://www.indiana.edu/~deanfac/sotl/eventsintro.html

http://www.carnegiefoundation.org/AboutUs/Staff/walker.htm

A Dedicated Director

A campus SOTL initiative needs a person charged with responsibility to take actions on ideas, to follow up actions to ensure completion, to convene groups, to plan and stage events, to coordinate faculty and administrative viewpoints, to “cheerlead,” and to serve as a stable point of contact. If the director is a faculty member, release time is essential. The SOTL initiative needs to be a significant part of the director’s portfolio.

Samuel Thompson

Director of the Campus SOTL Initiative

E-mail to Samuel

• Insure Representation Across Disciplines

• Tap Respected Faculty

• Let FACULTY Define Initiatives

• Support FACULTY Initiatives

Success may depend on the initiative being of, by, and for the faculty. Advisory council members can steer the enterprise. They can be inspired by the prospects of what the initiative can do and so contribute their own time and effort without acknowledgement in tasks which they, themselves, often define. This is “bottom up” infusion of energy.

Advisory council members should be chosen carefully. Ensure membership is representative. Primarily use senior, respected faculty: they know the ins and outs of the institution and their presence validates and lends prestige to the SOTL effort.

Make sure the initial SOTL thrusts on the campus reflect consensus of faculty on the advisory council. Let faculty members feel empowered to influence key decisions. Advisory council members suggested doubling the amount of the SOTL grants proposed by administration, defined and addressed the human subjects issue for SOTL, and took the initiative to bring the research establishment into partnership.

Administration supported the advisory council initiatives and decisions.

The Faculty Advisory Council

A Core of Faculty Members Willing To Engage in This Work and Share It With Others

Because this unit of the tutorial deals with launching campus SOTL initiatives, much emphasis is on administrative considerations. However, all the administrative brilliance in the world will not attract or sustain faculty in a SOTL initiative if they do not see relevance to academic issues important to them.

If a campus already has scholars of teaching and learning, these may provide examples or kernels of issues which engage colleagues. But what if such scholars and issues are not readily identifiable?

Engaging faculty should not be a struggle. The vast majority of faculty have a natural curiosity about teaching issues and a latent desire to do teaching-related inquiry. Making time and resources for reflection and inquiry available to faculty in light of their teaching loads and other responsibilities might be a challenge but as Shulman has suggested, students already provide faculty with more information than they normally use. Thus additional work, if any, in information-gathering should be minimal.

A possible two-step procedure is to conduct a kind of “needs assessment” … formal or informal … to identify campus issues for possible investigation; e.g. academic incivility. The second step is to connect an identified need to language and issues that people care about … framing the issue to make it attractive in the particular setting.

If engaging faculty in issues and framing them is of concern to you, you might keep this concern in mind when viewing other examples of campus programs , the issues investigated in Faculty SOTL Projects, and Framing the Question.

• Get Out the Crowd to Initial Events

• Showcase Campus Scholars

• Identify Campus-wide Needs and Interests

• Partner with Stakeholders

Identify and encourage faculty who wish to pursue scholarly inquiries into teaching or have already done so. Make these visible as models and mentors in high profile events rather than bringing in outside speakers.

Kick off the SOTL initiative with a “Celebration of Teaching.” Send personal invitations to winners of teaching awards, recipients of teaching-related grants, teachers of pedagogy courses for graduate students, and all others who have demonstrated interest in teaching in an identifiable way. Send personal invitations also to deans, chairs, and P&T committee members. About 200 faculty showed up for our Celebration, more than in anyone’s memory for a teaching-related event. Personal attention has continually proved important in building a SOTL community “one faculty member at a time.”

Use information-gathering opportunities to learn the needs, goals and activities of people and units across the campus. How can SOTL help to fill their needs? Focus on issues for investigation that cut across disciplinary boundaries.

Seek to make the SOTL initiative inclusive of all academic and non-academic units on the campus. Seek to involve undergraduate and graduate students, professional staff in academic affairs and student affairs, librarians, and faculty in all seasons of academic life. Seek as well to support and cooperate with all other teaching-related initiatives rather than to compete with them. Think of ways and themes for collaboration that advance the goals of stakeholders as well as those of the SOTL initiative.

Getting People Involved

Getting People Involved:An Example

As an example of partnering with stakeholders, consider librarians. They generally see themselves as academics but do not always feel that others see them in that light. When invited to join and help grow the SOTL initiative, they were very pleased for the opportunity to participate with faculty in an obviously academic enterprise. The librarians enriched SOTL with talent, initiatives, and resources.

Librarians:

•volunteered as consultants to individual faculty investigators needing assistance in locating unfamiliar teaching and learning literature.

• gave presentations to show use of information technology to access SOTL-related resources.

• made free document delivery systematically available to scholars of teaching.

•acquired multiple copies of critical books and put them on reserve for “SOTLites”.

•served on working committees of the Faculty Advisory Council.

The librarians, in turn, found participation rewarding both individually and collectively.

Getting People Involved (continued)

This campus model as well as all six campus SOTL models described later in this tutorial draw direction and resources from AAHE and Carnegie national initiatives … and frequently publicize the national connections to the local community. Other connections to related national initiatives include Peer Review, Problem-Based Learning, and Preparing Future Faculty.

•Exhibit Administrative SupportIt’s important not only to have administrative support but to exhibit it. One way is by regular participation of key administrators in SOTL events. Serving food exhibits institutional backing with funds as well as increasing participation, especially for events held at lunchtime or the end of the working day. Examples of SOTL events

•Provide Scaffolding for FacultyIf a SOTL initiative is to achieve it’s objectives, getting faculty interested or involved is not enough. Many need “scaffolding” … various forms of support … on the pilgrimage from initial interest to completion of scholarly work. Planning for this scaffolding and building it into the campus model is essential.

•Connect with National Initiatives

http://www.indiana.edu/~deanfac/sotl/eventsdescr.html

• Change the faculty member’s view of what faculty members doUnder the aegis of SOTL, employ administrative resources to acknowledge and reward a fuller range of faculty work.

• Expand the range of activities that get rewardedMany campuses have faculty who have done scholarship of teaching for years. However, nobody necessarily knows who they are if such scholarship does not count in reward systems. In this campus example, small grants ($1000) were offered at the start of the SOTL initiative for scholars of teaching to present work already completed or well underway. These unusual “presentation” grants for work that may have already been done made existing scholars and their projects visible campus-wide as models. Later conventional “research” grants to undertake new investigative projects were offered.

Many campuses already have small grant programs. These can be tweaked to foster SOTL. A spin-off benefit can be greater productivity for a given institutional investment in grants; e.g. a grant that was previously to implement a teaching innovation will, as a SOTL project, also require data-gathering and analysis to assess the effect of the innovation as well as dissemination of results to peers.

• Expand the range of rewardsOpportunities to present their work to large interdisciplinary audiences in high profile campus-wide events; to have their abstracts, biographical sketches, and photographs widely disseminated; and to experience the support and confidence of the central administration proved enough reward to some faculty members for them to offer themselves as SOTL presenters(without grants). For some faculty members, finding a community that valued their work to improve teaching and that helped them find opportunities for professional growth and recognition beyond the campus and beyond opportunities available in their departmental and disciplinary cultures also proved rewarding.

Changing the Institutional Culture

IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC YEAR 1999-2000

TEACHING ACTIVITIESA. Courses taught (weekly contact hours reported by course number in

tabular form).B. Development or major revision of course(s) during the year.C. Dissertation, Research and Field Work CommitteesD. Teaching awards and honors, including those of your students.

Indiana University’s summary report form for individual teaching activities required of each faculty member annually was as shown above prior to Fall 1999. It is similar to that used in many institutions. There are also sections (not shown) for Research & Creative Activities and Service.

Look at the categories of reportable teaching activities. Category B is generally taken to mean curriculum development or major revision of curriculum. Activities associated with improvement of pedagogy or with demonstrable evidence of improved student learning are not encouraged on such a report. Indeed, the lack of a place to report pedagogical activities on the form is a tacit statement about institutional values and rewards.

Changing Institutional Culture: One Example

TEACHING ACTIVITIESA. Courses taught (weekly contact hours reported by course number in

tabular form).B. Activities directed at improving instruction, learning, or course

administration. (Please describe rationale for/description of innovations, methods/measures for assessing outcomes, and results.)

Please note: Scholarly activity related to teaching and learning (e.g. investigation/research, dissemination/publication of results) should be reported under the section on Research/Creative Activities.

IU FACULTY SUMMARY REPORT (REVISED) IN ACADEMIC YEAR 1999-2000 AND THEREAFTER

In Academic Year 1999-2000, the Faculty Summary Report form for the Bloomington campus was revised to include the new Category B shown in red. The former categories B, C, and D respectively become C, D, and E on the new form. This

revision was prompted by the SOTL initiative.

Changing Institutional Culture: One Example (cont.)

Changing Institutional Culture: Another Example

An initiative of Craig Nelson, a faculty member committed to the campus SOTL initiative and a Carnegie Scholar, led to an institutional policy of partially funding travel of faculty members to attend teaching-related conferences. Candidates for support must show demonstrable connection to their scholarly interests in teaching and learning and provide a quid pro quo to the campus in return.

Craig Nelson is a co-founder of the campus SOTL initiative. His contributions are many. It’s very helpful to have a teacher-scholar of

his prominence involved! Craig Nelson

Biology

E-mail to Craig

mailto:[email protected]

Changing Institutional CultureFurther Examples

Input from the Faculty Advisory Council and interested faculty researchers influenced the human subjects process to facilitate SOTL

activities.

Input from the Faculty Advisory Council and interested faculty researchers facilitated access to institutional data (registrar, etc.) for SOTL

researchers.

These achievements were important steps in “developmental scaffolding” for faculty embarking on SOTL projects. An aversive institutional review board decision regarding students as human subjects or inability to gain access to essential background data regarding the students in the study can be lethal to a SOTL project.

Campus Participation in SOTL Program of Events

David Pace provides an overview of SOTL program impact.

14 main events in 1999-2000 program; 11 in 2000-2001 – about one every two weeks in fall and spring terms

108 campus units represented ( 74 academic departments/ programs)*

Participants:

496 individual faculty members

286 tenure/tenure track (22% participation rate for T/TT)

51 individual staff

237 graduate students

Total: 784 individuals (63% faculty, 30% graduate students, 7% staff)*

Total attendance: 1598 (53/event in 1st year and 77/event in 2nd)**Greatly exceeding expectations of many observers

Assessing SOTL Program Impact:Participation Statistics 1999-2001

Selected Faculty Comments about the Campus SOTL Initiative

We need SOTL

The events I attended were terrific. Keep it up. And keep me in the loop.

I was not able to participate directly in most SOTL events, due to numerous personal and professional obligations. But I was impressed by the degree to which SOTL spurred conversation on teaching among my colleagues. I also met with one SOTL participant to consult on his research project, and I was impressed with the enthusiasm he was bringing to his teaching and the related project. SOTL has been great for this campus (and, more importantly, our students), and I plan to participate enthusiastically next year. Thanks!

SOTL is an outstanding source of intellectual support for the graduate school's efforts in "preparing future faculty."

I realize that the program is in some sense intended to support and provide a forum for people producing the scholarship of teaching and learning. However, please do keep in mind how useful it can be to those of us simply working on teaching.

I think you are doing a wonderful job, and I fully support your work

SOTL is a very valuable asset to the university and the School of HPER. Please continue with your excellent efforts.

I think you are beginning to make substantive

improvements to the learning environment and are slowly raising awareness in the general faculty about the importance of teaching and how this may be approached through scholarship.

I am afraid I have a rather dim view of "scholarship" about teaching or learning. Teachers should teach, learners should learn. Scholarship should be reserved for inquiry in genuine subjects.

Scarce resources would be better devoted directly to teaching (e.g. smaller class sizes) or traditional substantive areas of research

The ratio of positive to negative comments is actually much higher than in the sample here.

Support for SOTL by research establishment provided on same basis as research in the disciplines (matching funds, summer fellowships, etc.)

Campus colleagues with specialized expertise voluntarily contribute time and effort to consult with scholars of teaching.

Growing constellation of linked efforts affording opportunities

and support (course portfolios, PFF, etc.)

Large informal “SOTL community” on campus Samuel explains why multidisciplinary community is important to SOTL (beginning at 2 minutes: 10 seconds into the audio clip)

Scholarship given birth under SOTL initiative begins to gain recognition (and invitations) beyond campus

A visiting scholar of teaching and learning, Professor E. F. Redish, Department of Physics, University of Maryland, remarked that he had not previously experienced such a multidisciplinary community of scholars investigating issues of teaching and learning.

After meeting members of this community, Lee Shulman, President of the Carnegie Foundation for the Advancement of Teaching, echoed similar sentiments ... that this SOTL community has a truly extraordinary robustness and interdisciplinary character.

Assessing SOTL Program Impact: Scaffolding

Think for a minute about your own campus. What administrative priorities or structures might help in your campus level SOTL program? What elements of the preceding example, if any, may be applicable? What hurdles might

need to be overcome on your campus?

Pause here to reflect on administrative features and obstacles on your own campus in the context of the Bloomington example. Press the button and share your thoughts. Your comments may prove useful in unforeseen ways. Besides, we’re REALLY interested!

TASK D: Refining Your Own Campus SOTL Program or Plan

mailto:

Other Campus Examples

• Elon College

• The Citadel

• Rockhurst University

• Abilene Christian University

• Notre Dame University

• Middlesex Community College

• Task E – Applicable Features

Campus examples of SOTL initiatives from six different institutions of widely varying types are exhibited in the following frames. All of these campuses are participants in the AAHE Campus Program

After surveying these examples, you’ll be asked to identify features of them that might be applicable to your campus program (Task E)

Elon College

• Multidisciplinary, Multiyear, $72,000 Investment – $6000 projects in each of 3 years

– Projects directed by faculty-student research teams

– Learning for BOTH student and teacher

– Eight projects selected in years 1 and 2

• Create intellectual engagement– New thinking in diverse fields – Application of learning to life – Opening spaces for reflective integration

The magnitude of the Elon investment in the SOTL projects is notable for a school with about 4000 students. The projects focus on conditions of learning and the individual course is the unit of study. The use of student-teacher research teams to carry out each project is a distinctive feature.

Elon jointly with Western Washington University won the first AAHE “Going Public” grant. John G. Sullivan, Powell Professor of Philosophy, kindly provided the information used in this tutorial. For more information

http://www.elon.edu/carnegie/

The Citadel• Mission

– Increased campus awareness of and participation in SOTL• Focus

– Communication, Resources and Continuing Education – Self-selected research projects

• Highlights– Biweekly, participatory meetings with assignments

• Effectiveness– 15% of full-time, tenure-track faculty at bi-weekly meetings– 12% of full-time, tenure track faculty in classroom research

• Administrative Support– Attendance at functions– Financial support

The Citadel SOTL initiative got a big boost from a Mimi Steadman workshop on classroom research. The Citadel SOTL website is rich in resources and ideas. It’s well worth a browse. One distinctive feature is the schedule of “assignments” for the year given to participating faculty.

Professors Alix Darden and Suzanne Mabrouk spearhead work in this initiative. Suzanne can be e-mailed through the website. Alix was selected as a Carnegie Scholar for 2001-2002. E-mail toAlix

http://www.citadel.edu/carnegie



Rockhurst University• Beginnings

– (Fall 1998) All University Symposium

– (Spring 1999) Follow-up Symposium

Year-long Carnegie faculty seminar

– (2000-2001) Carnegie faculty seminar continued

• The Rockhurst “Carnegie Seminar”– Central Questions

– Seminar Members

– Discussions

– Methods

• Products– Formal Letter on SOTL

– Faculty SOTL Projects

• Selected Key Issues and Observations– Obstacles to Discussion

– Interdisciplinary/Collaborative Approaches

– “Where's the beef?”

– “Scholarly Teaching” as best first path

– To be a good consumer of the SOTL

Rockhurst, a small (3000 students) independent university in Kansas City, conducted a year-long Carnegie seminar which resulted in a formal letter addressing reward for scholarly teaching and scholarship of teaching in tenure and promotion.

An observation of the Rockhurst Carnegie Seminar … which is entirely consistent with observations at other institutions … is that a focus on scholarly teaching is a good first step to scholarship of teaching.

Several faculty research projects are underway at Rockhurst. One researcher is Professor Anita Salem (mathematics), also a Carnegie Scholar of Teaching. Anita provided the information for this tutorial. E-mail to Anita


Abilene Christian University

• 19 Faculty Engaged in SOTL Projects

• Strong Institutional Support

– Stipends for Materials & Resources

• Travel to Teaching-Related Conferences

-Ongoing Peer Meetings,Videoconferences

Abilene Christian provides an example of an effective SOTL initiative at very modest cost in a private institution with about 4500 students. Faculty participants make a year-long commitment to get their projects to the stage of publication. They are given small stipends for materials and resources. An outside consultant provides a day long workshop to help shape the projects. Ongoing support is provided through peer meetings and videoconferences with the consultant.

Visit the ACU SOTL website or request a SOTL program brochure from Nancy Shankle, English Department Chair. Nancy steers the SOTL initiative at ACU. E-mail to Nancy

http://faculty.acu.edu/~shanklen/softl/home.html


University of Notre Dame Initial campus conversations with 90 campus leaders

– SOTL needed support

– RFP resulted in 9 funded SOTL projects

Sample research question: Do new teaching methods in introductory engineering affect students’ learning?

Support for SOTL teams includes these elements:– $5,000 per team for student time, equipment, supplies, faculty time

– Consulting with methodology experts

– Group meetings 2x/semester for mutual support

– Help in dissemination of results

The Notre Dame SOTL initiative features faculty research projects, campus conversations, and a Carnegie Scholar. Barbara Walvoord and her team of “experts” provide support to investigators in regional institutions as well as at Notre Dame E-mail to Barbara

Notre Dame’s approach – to develop a community of scholars and define SOTL by example – is similar to the approach in Bloomington. The impediments are similar too … like faculty reluctance to engage in a scholarship for which they feel they have no training.

http://kaneb.nd.edu/pages/sotl.shtml


Middlesex Community College

Interdisciplinary group of 9-10 faculty

Receive 1 course release for minimum 2-year commitment

Meet biweekly for two-hour seminars

Study SOTL - and how to create environs that foster intrinsic motivation

Undertake individual faculty projects

A goal at Middlesex is to link faculty projects to intrinsic motivation in students. Professor Donna Duffy (psychology), also a Carnegie Scholar, provided information about the Middlesex SOTL initiative.

Donna’s own project at Middlesex is ‘Resilience as a Path to Integration in Abnormal Psychology.” She chose resilience, a positive attribute, as a theme because abnormal psychology is generally about negatives … the problems people face. E-mail to Donna

What’s common to all of the seven campus examples (including Indiana University)?

-they involve institutional investment and support for the SOTL program

-they are connected to AAHE/Carnegie national initiatives.

-they involve faculty investigative projects into issues of teaching and learning


Task E: Campus Examples of SOTL Initiatives

List features of the examples you have just seen that might be most applicable to your campus.

Please reflect on the applicability of the campus examples just discussed to your own institution. Construct a “mock-up” of a SOTL program on your campus, drawing on the features of programs at other campuses. Press the button on the frame and send your model to us.

mailto:

Unit 2B Faculty SOTL Projects

Donetta Cothran – Kinesiology

Valerie O’Loughlin – Medical Sciences

David Pace – History

William Becker - Economics

Rita Naremore – Speech and Hearing Sciences

Dennis Jacobs – Chemistry

Leah Savion – Philosophy

Other Examples – Carnegie Scholars

Task F: Minute PaperAfter viewing these projects, you’ll be asked to identify the most important points for you in the first four modules of the tutorial (Task F)

Donetta Cothran - Kinesiology

For Donetta Cothran, a junior faculty member, interest in improving her students’ sense of fairness in grading of their papers led naturally into a scholarly project.

Donetta gained insights about scholarship of teaching from her participation in a campus SOTL initiative and involvement in her own scholarly project.

E-mail to Donetta

Expectations and Effects of Graded Writing Assignments

Fostering Interactive Learning in a Large Science Course and

Methodically Measuring the Effects

Valerie Dean O’Loughlin

Medical Sciences

A Neophyte’s Adventures in the Scholarship of Teaching

Valerie’s innate interest in teaching found form and expression in the campus SOTL initiative. Her project is a “Child of SOTL”, conceived, born, and nurtured to scholarly product within the campus initiative.

“Scaffolding” facilitated her acquisition of the concepts and

resources needed to carry out her study. Valerie’s initial presentation of her work to peers was of interest to many both for the pedagogical techniques she employed and for her means of assessing their effects.

Valerie describes the launch of her project

Powerpoint presentation of the complete study

E-mail to Valerie

Details of Valerie O’Loughlin’s study are considered later in the tutorial in Questions, Designs, and Resources.

What do I want my Students to be Able to Do?

David Pace–History

David Pace, a Carnegie Scholar of Teaching, describes his odyssey to the question at the heart of his scholarship; “What are students supposed to be doing (or learning) in my course?” He shares his realization that the above question led to understanding of his discipline in ways he never understood it before.

E-mail to David

Bill Becker’s project is an example of a study that is not centered on one’s own classes. Rather by studying selected published papers of others, he seeks to separate empirical results from conjecture about the student outcomes associated with classroom assessment techniques and other teaching strategies designed to engage students actively in the learning process.

In this project, Bill also seeks to advance the scholarship of teaching and learning by establishing an eleven-point set of criteria which faculty members can use to evaluate inferential studies.

Bill’s study culminated in a paper: What Does the Quantitative Research Literature Really Show about Teaching Methods?

E-mail to Bill

William Becker Economics

What Does the Quantitative Research Literature Really Show about Teaching Methods?

The problem: a group of faculty members concerned that teaching excellence was getting short shrift in the faculty evaluation process

The initial question: What can (and do) student evaluations really tell us?

Rita Naremore

Speech and Hearing Sciences

The department had a culture that valued excellent teaching, and in some ways took it for granted. Salary increases were based on data provided using the campus faculty annual report form It had become abundantly clear that those faculty members engaging in innovative or particularly effective pedagogical change were not being rewarded because nobody knew what they were doing or how effective it might be.

Rita introduces the project

Powerpoint presentation of complete study E-mail to Rita

A Departmental Level SOTL Project

Peer evaluation presented the group with a much larger problem. After much heated discussion, it became clear that the occasional visit to a classroom by one’s colleagues did not seem an adequate form of peer evaluation. This faculty agreed that what was missing was a systematic review of course organization and materials (including exams, visual support for class presentations, etc.) conducted by people outside the department who knew the content.

Once the question regarding student evaluations seemed answered, attention turned to peer

evaluation.

The faculty consensus: traditional classroom visitation is not very revealing of anything useful. What we really need is evaluation of course structure and materials used in the class for teaching and evaluation. We’d like to have periodic external peer review of these materials.

A Departmental Level SOTL Project Issue # 2: Peer Evaluation

• The biggest concern: IT TAKES TOO LONG TO PUT ALL THIS STUFF TOGETHER!

• What are we doing? We’re working toward the development of course portfolios, to be done over time and turned in by faculty members every second or third

year, not every year.

Although agreement in principle was quickly attained, agreement in practice was not. Several faculty members remained convinced that they did not have time to put together their teaching materials in a way that made them accessible to an outside reviewer, especially if they had to explain what they were trying to do. Nonetheless, the department is continuing to work toward a system of course portfolios, along with a checklist that can be used by outside reviewers who are looking at the portfolios.

A Departmental Level SOTL Project: Ongoing Concerns with Peer Evaluation

A Departmental Level SOTL Project: The Campus and National Ties

Review the Relationship of Peer Review of Teaching, Course Portfolios, and SOTL

Visit the Indiana Campus Course Portfolio Initiative

More Information About the Consortium Project Led by Nebraska

E-mail to Simon

The connection to course portfolio projects exemplifies the way SOTL may be synergistically connected to and integrated with other teaching-related initiatives.

Simon Brassell Geological Sciences Chair of the Campus

Course Portfolio Initiative

The departmental course portfolio effort should eventually tie in with a campus-wide effort now underway, conducted by an interdisciplinary team in conjunction with a national project funded by Pew, based at the University of Nebraska, and involving four other research universities.

http://www.indiana.edu/~deanfac/portfolio/index.html

http://www.unl.edu/peerrev

http://www.unl.edu/peerrev

An Alternative Approach to General

Chemistry

Assessing the Needs of ‘At-Risk’ Students with Cooperative Learning Strategies

Dennis JacobsProfessor of ChemistryUniversity of Notre Dame

The next 7 frames contain details of this project. The project exemplifies further the tie between SOTL and course portfolios as well as providing an excellent example of each! Dennis Jacobs is a Carnegie Scholar of Teaching and Learning.

The project in complete detail (log-in required)

Skip to the next SOTL project.

E-mail to Dennis

An Exemplary Course PortfolioAnd

A Superb Model of SOTL

http://kml2.carnegiefoundation.org/kml/login/

QuickTime™ and aGIF decompressor

are needed to see this picture.

- 4 -

Rationale:• Recognized Problems:

• ‘At-risk’ students (Math SAT ≤630): • dropped out of General Chemistry. • didn’t take any advanced science.• frustrated by large lecture format.

• Alternate Course Design:• Similar requirements and lectures.• Comparable exams.• Various activities involving structured cooperative

learning.• Initial Comments on Proposal

• “Only delaying inevitable failure.”• “Efforts should be focused on the best not the ‘at-risk’

students.”

Comments that colleagues initially made to Dennis are shown in the last bullets. Dennis proved them wrong. He demonstrated convincingly that ‘at risk’ students could perform at the same standards as other students in general chemistry and go on to succeed in organic chemistry (a sequel course) at the same rate.

An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)



- 5 -

Documentation:• ‘At-risk’ students markedly more likely to drop or fail

the course in the traditional class format.

As you view this project,, note Dennis’ methodologies and the thoroughness of assessment. This is a well designed deed of scholarship. Ask yourself at the end, whether you feel convinced from this brief slide series of the impact of his alternative course for ‘at risk’ students.


Implementation:• Aim:

• Provide improved learning opportunities for ‘at-risk’ students.

• Develop more effective teaching in large lecture format.

• Alternate Course Design:• Introduced opportunities for structured

cooperative learning including:• discussion of concepts in pairs.• small group in recitation sections.• work as pairs in laboratory.

• Mandatory recitation sections:• more time committed to class. • direct contact with instructors.



- 6 -


Traditional vs. Alternative Classes:• Similarities:

• Size (250 students), text, chapters.• Lecture time (3 hr), lab time (2.5 hr).• Lecture format (Powerpoint slides and demonstrations).• Exam format and many exam questions.

• Differences in Alternative Section:• Mandatory recitations (1 hr/wk, 20 students);

attendance 95% vs. 10%.• Weekly homework (10 vs. 30, graded).• On-line quizzes (www chapter reviews).• Weekly feedback from homework, group problems, on-

line quiz, in-class questions.• Personal contact with instructor and follow-up if

performance declined.



- 7 -


Impact of Alternative Section:• Assessment Strategies:

• Effects on conceptual understanding, problem-solving and self-confidence:

• feedback from students.• evaluation of individual elements of the

cooperative learning activities. • Immediate and long-term benefits:

• retention of ‘at-risk’ students.• success in advanced science classes.

• Data Collection:• Recording in-class learning activities.• Tracking individual grades and progress.• Longitudinal study of ‘at-risk’ students:

• progress in subsequent classes.



- 8 -


Measurement of Impact:•Success of ‘At-risk’ Students:

•Better grades in General Chemistry.•Improved retention in class.•Higher success rate in subsequent classes.



- 9 -




- 11 -

Longitudinal Impact (follow-on course):

Intervention introduced between 1996 and 1997

Organic chemistry is a follow–on course to general chemistry. In 1997, a higher percentage of ‘at risk’ students who had taken Dennis’ alternative general chemistry course completed organic chemistry than the percentage of students in the traditional course.


Investigating Pet Theories and Naïve MisconceptionsLeah Savion’s personal interest in learners’ cognition and interference effects of naïve misconceptions caught on with faculty colleagues (the issue crosses most disciplines) and led to a cross-disciplinary study.

Leah explains misconceptions and their effects

A brief overview may be gained from a one page summary of the project

Add student misconceptions from your own experience to Leah’s Incomplete Inventory of Misconceptions

Further detail may be gained from her working paper

Leah warmly invites your peer review and comment (E-mail to Leah)

“What force acts on a coin that has been tossed up in the air”? The large majority of students who completed a course in mechanics in MIT gave the same wrong answer as the totally untrained students, citing the “original upward force of the hand”. The pet theory of motion at work here is that there’s no motion without force.

Pet theories are involuntary explanatory constructs that we all build from a very early age, in an attempt to understand the world around us, to build causal connections between events, and to enhance our sense of control over the environment. These theories are about every aspect of life. They are amazingly universal, are based on surface features of the relevant data, fragmented, inconsistent, contain principles that emerge spontaneously, complex, intricate, serviceable, and seemingly well organized. These robust theories are not normally tested against scientific, social or logical facts. A specialized psychological architecture that contains principles of quick reasoning, rapid accumulation of information, and economy of cognitive operations, may underlie and guide their construction and their perseverance in face of contradictory evidence.

Pet theories inevitably imply what I termed here “naïve misconceptions”, that prove extremely resistant to change. When the naïve meets the academically acceptable theories we deliver in class, the better students either attempt to alter the new information, or combine the incompatible principles, or, most commonly, adapt the new theory as a “school bound” explanation, not applicable in everyday life.

Leah Savion Philosophy


Scholarly Projects of Faculty (Continued)

-Framing the Question

-- Context of the Project

-- Gathering the Evidence

-- Emergent Findings

-- Conditions for doing SOTL

-- Benefits of the Work

-- Lessons Learned

Opening Lines may be a useful source for identifying issues of teaching and learning in one’s practice and framing them. Many faculty will be able to identify with the thoughts and experiences of the authors of these case studies. For faculty members who feel that they do their “thinking about teaching” in isolation within their departments or institutions, this publication may feel like a conversation with colleagues.

Where you can learn about additional scholarly projects. Opening Lines contains 8 case studies of approaches to individual projects by Carnegie Scholars. The seven bullets below form the outline of the narrative in each case study. Style and tone are similar to those in the video vignettes used in this tutorial.

Opening Lines comes with a CD containing a variety of tools and resources for scholarly projects. These include interviews (some as video clips), survey instruments, protocols, and reference sources. The publication showcases recent work in CASTL

"MINUTE PAPER"

1. What were the 3 most important points presented in Units 1A, 1B, 2A and 2B of this tutorial?

2. What 3 things would you most like to learn in the remaining two modules, Units 3A and 3B, of the tutorial?

.

This Classroom Assessment Technique (CAT) denotes the end of Unit 2B. The formative feedback offered by you to us here will help make this a better tutorial for everyone. Please press the button and send your feedback. If you’ve worked through the tutorial continuously to this point, you might want to take a break before entering Unit 3 - the last stretch of the continuum from alpha to omega (and it’s more than a third)!

Task F: Looking Back Over Travel Along the Continuum Thus Far

Unit 3A Bridges to Productivity

How Could I Do Scholarship of Teaching and Learning (Genres of SOTL)

Task G: Reflecting on Genres

Approaches to Scholarship Via Classroom Research Projects

Where to Publish and Present

Sources of External Funding

Task H: Reflecting on Classroom Research and Key Reference Books

Unit 3 may be thought of as a rudimentary “tool kit” for embarking on scholarly projects in issues of teaching and learning. This is the longest unit. 3A surveys some general considerations. 3B focuses on project design.

Still Another Look at What We Mean by the Scholarship of Teaching and Learning

Before venturing into a mélange of categorization schemes, it may be well to pause for another overview of this scholarship.

Do the following situations represent scholarship of teaching and learning?1. A faculty member reforms a course, building into the reform all three criteria of scholarly teaching.

She puts the course on-line not only for students but for all faculty in the discipline, and furthermore invites faculty to peer review and critique it on the web. The course catches the attention of faculty at another institution. They are interested in it and invite the course author to come there and give a colloquium on the course. The author does so and reflects on the attributes of the course for her students. Subsequently, the department in the second institution adopts large elements of the author’s course to improve their own undergraduate program.

2. An assistant professor in quest of tenure develops a course portfolio as evidence of his effectiveness in improving student learning. The promotion and tenure committee routinely sends candidate packages to outside reviewers. External reviewers comment on the excellence of this portfolio as convincing evidence of the candidate’s teaching effectiveness. The candidate is granted tenure. The course portfolio is subsequently disseminated through the state university system as a model for other faculty.

Still Another Look at What We Mean by the Scholarship of Teaching and Learning (Cont.)

If you accept Shulman’s quasi-definition , you will probably conclude that both situations in the previous frame DO represent scholarship because they satisfy all three of his conditions. As key reformers urged, the concept of scholarship is broadened to include useful contributions that do not necessarily conform to conventional scholarly forms in the disciplines. To be sure, scholarship of teaching and learning includes publication in journals, but it also includes other forms, as in the two previous examples, that contribute to the advancement of teaching and learning both directly and indirectly.

To survey possibilities for this scholarship, we exhibit several categorization schemes. First, we’ll consider Craig Nelson’s “Genres of SOTL.” Then we’ll look at “traditional and classroom research” and “qualitative and quantitative methods.” All of these schemes are messy and inconsistent. The genres, for example, sometimes seem separated by unit of analysis at other times by type of research design. And as we see from the two examples in the preceding frame, they are not necessarily inclusive of all possible forms of scholarship. Similarly, much good scholarship combines elements of traditional and classroom research as well as qualitative and quantitative methods.

Well, why do we even bother with the classification schemes if they are inconsistent? Because they give us valuable “points of possibility.” They may be thought of as topographical features that map the territory. The actual point at which one locates one’s individual deed of scholarship is usually somewhere between the “points of possibility.”

Different Genres of SOTL

– Reports on Particular Classes

– Reflections on Years of Teaching

– Larger Contexts—Comparisons

– Formal Research

– Meta-AnalysesGenres of SOTL overlap and can be combined or subdivided variously. The particular examples of scholarship for each genre are chosen to illustrate importance for improving teaching and learning and largely excerpted from Craig Nelson’s paper on genres: "How could I do scholarship of teaching and learning?"

Two opening points:

1. Learning and teaching are complex activities where approximate, suggestive knowledge can be very helpful, and, indeed, may often be the only kind that is practical or possible

2. Much important expertise on teaching resides in the day to day practices of good faculty. Typically, this knowledge remains private and is totally lost when its possessor retires. A key task in this field is systematically making much more of this expertise public.

Craig Nelson introduces the genres of SOTL

How Could I do Scholarship of Teaching and Learning?

Reports on Particular Classes

•It Worked! Important pieces of our expert knowledge as experienced practitioners can be preserved by writing up exemplary approaches to content or pedagogy that work especially well in our own classes. In this genre, the teacher's own impressions of the effectiveness frequently serve as sufficient assessment. The trend now is to try to document the effectiveness a bit more formally using CATs and CR.

•Before & After: Assessments of Changes in Practice

-Qualitative The many examples of this genre in Angelo and Cross include a calculus class (pp. 69-72) in which the professor wanted to help students improve their problem solving skills. This example illustrates the process of refining the pedagogical questions and the successive modifications that are often necessary to make new pedagogical approaches work successfully. In this case, the new pedagogy improved student success sufficiently that no student made an F, despite the maintenance of high academic standards

-Quantitative Group study approaches to calculus decreased the D/F/W rate for African Americans from 60% to 4%. See R. E. Fullilove & P. U. Treisman. 1990. Mathematics Achievement Among African American Undergraduates at the University of California, Berkeley: An Evaluation of the Mathematics Workshop Program. Journal of Negro Education 59: 463-478.

How Could I do Scholarship of Teaching and Learning?•Different Genres of SOTL

–Reports on Particular Classes–Reflections on Years of Teaching–Larger Contexts—Comparisons–Formal Research–Meta-Analyses

Reflections on Years of Teaching

•Essays Developing Good Ideas An example providing good articulation of the rationale for SOTL is L. S. Shulman. 1993. Teaching as community property: Putting an end to pedagogical solitude. Change 25: 6-7.

•Summaries of Expert Knowledge Gained by Self-Reflection and Experimentation Examples are P. Frederick. 1981 The Dreaded Discussion: Ten Ways To Start. Improving College & University Teaching 29:109-114. P. J. Frederick. 1986. The Lively Lecture--Eight Variations. College Teaching 34:43-50 Many course portfolios fit here too. Path breaking examples can be accessed through links at Implementing Entities: Carnegie and Pew. Another example by Randy Bass is (http://www.georgetown.edu/bassr/portfolio/amlit/ .

•Integration of Larger Frameworks with Classroom and Curriculum Practice J. D. Herron. 1975. Piaget for Chemists: Explaining What "Good" Students Cannot Understand. Journal Chemical Education 52:146-150. One factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable in all quantitative fields. R. J. Kloss. 1994. A nudge is best: Helping students through the Perry scheme of intellectual development. College Teaching 42:151-158. Another factor that explains why bright, hard-working students can do poorly and how we can help them. Easily applicable across the curriculum.



http://www.georgetown.edu/bassr/portfolio/amlit/









Larger Contexts: Comparisons Across Courses and Student

Change Over Time •Qualitative Studies to Explore a key Issue Wm. G. Perry, Jr. [1970] 1998. Forms of Intellectual and Ethical Development in the College Years, A Scheme. New introduction by Lee Knefelkamp. Jossey-Bass. The impetus here was the observation that students could flunk out of Harvard despite working quite hard at learning the course material. The longitudinal design used extensive interviews with students at the end of each of their four undergraduate years. Patterns of intellectual development were inferred and checked for inter-judge reliability. A very influential study.

•Quantitative comparisons of Different Courses or Sections M. D. Sundberg & M. L. Dini. 1993. Science majors vs nonmajors: Is there a difference? Journal of College Science Teaching. Mar/Apr 1993:299-304. Question: Does covering more teach more?. Both courses taught with traditional pedagogy and by multiple instructors, but with different intensities of 'coverage.' Learning assessed with the ACT exam for AP Biology "The most surprising, in fact shocking, result of our study was that the majors completing their course did not perform significantly better than the corresponding cohort of nonmajors."

•Comparisons over an Array of Courses with a Common Assessment Instrument R. R. Hake. 1998. Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics 66: 64-74. (http://carini.physics.indiana.edu/SDI/welcome.html#z44). Uses a common test of Newtonian physics to compare increases in understanding achieved by a wide array of pedagogies in introductory physics courses at institutions ranging from high-schools to Harvard. Found that "interactive engagement" approximately doubles the amount of physics earned. An especially important model for emulation in other disciplines.



Formal ResearchHow Could I do Scholarship of Teaching and Learning?•Different Genres of SOTL


Experimental Analyses C. M. Steele. 1997. A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist 52:613-629.

Classroom Incivilities R. Boice. 1996. Seminal study on incivilities. Identifies and counts perceived incivilities committed by faculty as well as students and investigates relation between them. Journal of Research in Higher Education 37: 453-485.

The Impact of One Minute Papers on Learning in an Introductory Accounting Course E. D. Almer, K. Jones, & C. Moeckel. 1998. Four hypotheses investigated simultaneously in one class using a fractional factorial design. Issues in Accounting Education 13:485-497.

The One-Minute Paper: Some Empirical Findings J.F. Chizmar & A. L. Ostrosky. 1998. Four teachers each taught control and experimental sections. Data were analyzed using a multiple regression model. Journal of Economic Education 29:3-10 This study like the one above on incivilities could also be categorized under “Larger Contexts” The last two studies of one-minute papers are good examples of formal research at the classroom teacher level. They are also important contributions to the structure of evidence that one-minute papers improve student learning.

What constitutes “formal”? Does “formal” reside in the lower right quadrant of Rice’s diagram? Not necessarily. We use “formal research” to mean forms of scholarship in which the principal goal is research. Usually such scholarship will incorporate one or more features of conventional research paradigms. Formal research can still be classroom-based and related to active practice (see the last three examples below).

http://www.indiana.edu/%7Eeconed/issues/v29_1/1.htm

Meta-Analyses

•Annotated Bibliographies R. N. Johnson, D. M. Enerson & K. M. Plank. 1996. Diversity: A Selected and Annotated Bibliography Center for Excellence in Learning and Teaching. Pennsylvania State University.

•Brief Annotated Summaries of Key Research Findings T. A. Angelo. 1997. The campus as learning community: Seven promising shifts and seven powerful levers . AAHE Bulletin 49:3-6.

R. B. Barr & J. Tagg. 1995. From teaching to learning: A new paradigm for undergraduate education.

Change 27:13-25.

•Formal (Quantitative) Meta-Analyses L. Springer, M.E. Stanne & S.S. Donovan. 1997. Effects Of Small-Group Learning On Undergraduates In Science, Mathematics, Engineering And Technology, A Meta-Analysis. National Institute for Science Education, University of Wisconsin. 608/263-4214 [average effect size "would move a student from the 50th percentile to the 70th..."]



RESPONDING TO THIS TASK IS NOT JUST BUSYWORK! Reflection on the “points of possibility” described in the genres, may be an important step in conceptualizing your own work in SOTL. You may want to read one or more of the example papers before attempting this task. Your conception of your own SOTL work need not coincide with one of the genres, of course!

Where does the course portfolio fit in the genres described?

Reflecting on the genres, which seems most appropriate for a scholarly project of your own? Which seems easiest for you to do?

Are there forms of SOTL for which the genres described do not apply? If so, how would you name one or more additional genres?

Task G: Reflecting on Genres

mailto:

•Classroom Research

Distinguishing classroom research from traditional research

A “grand schematic” of classroom research

Classroom Assessment Techniques

Effective Grading

The Course Portfolio

Task H: Reflecting on Classroom Research

Classroom research is described. A schematic diagram is exhibited and used to show strengths of various useful books. Finally, in Task H you will be asked to identify the step in classroom research that seems easiest and the reference book that seems most useful.

Approaches to Scholarship Via Classroom Research

“Classroom Research is not traditional research conducted in or on classrooms. It is a specific methodology designed for discipline oriented teachers without training or experience in the methods of educational research. Classroom Research is ongoing and cumulative intellectual inquiry by classroom teachers into the nature of teaching and learning in their own classrooms. Inquiry into a question about how students learn typically leads to new questions and thus to continual investigations through classroom research.” Cross & Steadman

“Finding that almost no relationship existed between research on learning and collegiate teaching practices (professors were either oblivious to such research or ignored it), K. Patricia Cross concluded that the research itself was at fault for failure to pay attention to actual classrooms. She argued that teaching and learning reforms could only occur if they were based on concrete classroom situations. Cross made “how to do it” her calling card.” M. Lazerson, U Wagener, and N. Shumanis. Teaching and Learning in Higher Education. p. 17 Change 32: 3 2000

Many journals that publish articles about teaching and learning, both within disciplines and across disciplines, invite articles based on classroom research.

The term “action research” is sometimes used synonymously with classroom research. The former dates from the work of Kurt Lewin in the 1940’s and may be defined as “Research carried out by practitioners with a view to improving their professional practice and understanding it better.” Borg, Gall & Gall. 1993 Applying Educational Research 3rd ed. Longman,: 390

Cross, K.P. and Steadman, M. 1996. Classroom Research:

Implementing the Scholarship of Teaching, Jossey-Bass

Classroom Research

Traditional Classroom

Origin State of present educational literature Professor’s teaching practice

Purpose Contribute to existing state of educational theory

Obtain practical knowledge applicable in limited circumstances

Requirement Specialized training / broad grasp of pertinent literature

Specialized training not essential but developmental scaffolding may be

Benefit Field and researcher Students and professor

Traditional and Classroom research are compared for the sake of characterizing two “points of possibility.” One is not advocated over the other and it’s an oversimplification to view them as independent. Traditional research is often performed in classrooms. Classroom research may be enhanced by traditional methods and measures, especially if the results are for publication in national journals. For example, use of an “off the shelf” assessment instrument with widely accepted validity may be preferable to a “home grown” test or survey with unknown validity. Using someone other than the person who determines grades to collect potentially sensitive data from students is also a traditional method that may enhance a classroom research effort (and make it compliant with the local human subjects policy!)

A key point is that classroom research is a good point of embarkation for the typical faculty member using materials at hand and without specialized training in educational research methods. Classroom research is usually directed at local teaching issues or questions of specific interest to the teacher.

“Classroom research is not about finding universal truth but rather contextualized truth.” Tom Angelo

“Traditional” and “Classroom” Research

Improving Teaching and Learning via Classroom Research

Improved Teaching and

Learning

Existing Scholarship

Recorded Observation

Summative Assessment

Formative Assessment

Institutional Information Base

Issue

Goal

Reflection

Analysis

IMPETUS

INFORMATION GATHERING

EFFECT

SCHOLARLYPRODUCTIVITY

Synthesis

APPLICATION

Our grand schematic of classroom research! Schema with similar features are common; We envision three major components to a research project; a motivating goal or issue, acquisition of information bearing on achievement of the goal or resolution of the issue, and effects identifiable from the juxtaposition of impetus and information. Sometimes the project involves an intervention, but not necessarily.

We argue that if the scholarly project is done well, teaching and learning will inevitably improve regardless of the results of the study. Why so? Because the classroom research by its very nature will at least improve attention to students and information they provide, thereby improving understanding and communication. Student performance is also positively motivated by evidence that teachers care enough to study them and the efficacy of their learning.

After the third major component in the above schema (Effect) the possibility of a scholarly product may emerge (curriculum materials, article, presentation, etc.) Such a product may enhance the institution’s information base as well as the existing body of scholarship.

Classroom Assessment Classroom assessment is systematic and formative• Class is the unit of measurement rather than the individual • Conditions of learning may be assessed rather than student performance. • Correct and incorrect are not the emphasis. • Unexpected rather than expected responses are often most useful.• Review Unit I material about classroom assessment/research

Angelo, T. & Cross, K.P. (1993) Classroom Assessment Techniques:

A Handbook for College Teachers (2nd Ed) Jossey-Bass


Learning






Issue

Goal

Reflection

Analysis

IMPETUS


EFFECT


Synthesis

APPLICATION

Defining features of classroom assessment are shown in the bullets.They are quite different from the features of summative assessment associated with traditional grading.

The book belowhelps faculty clarify their teaching goals, explains formative assessment to improve achievement of goals and provides a catalog of 50 classroom assessment techniques (CATs).

Effective Grading Primary Trait Analysis (PTA)… building scales that make performance criteria explicit in order to:

-Categorize/classify student work. -Benchmark student learning and document changes. -Improve validity of grading.

Walvoord, B. and Anderson, V. (1998) Effective Grading: A Tool for Learning and Assessment, Jossey-Bass .


Learning






Issue

Goal

Reflection

Analysis

IMPETUS


EFFECT


Synthesis

APPLICATION

The book below:

Is a useful resource for classroom researchers

Aims at documenting how well students are achieving cognitive objectives and at providing evidence for change in achievement over time

Treats almost any conventional objection or concern about grading

Improved

Teaching and Learning






Issue

Goal

Reflection

Analysis

IMPETUS


EFFECT


Synthesis

APPLICATION

“I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began to lead me in the direction of something I had not seen anyone else doing: a portfolio that focused on the course rather than on all of one’s teaching. Being a social scientist, I began to think of each course … as a kind of laboratory - not a truly controlled experiment of course but as a setting in which you start out with goals for student learning, then you adopt teaching practices that you think will accomplish these and along the way you can watch and see if your practices are helping to accomplish your goals, collecting evidence about effects and impact.”

W. Cerbin quoted in Hutchings, P. (Ed.) 1998 The Course Portfolio, AAHE

As a kind of laboratory notebook for recording data and observations over time, the course portfolio keeps the teacher’s experience from being lost with the teacher’s memory. It’s a possible means of capturing “wisdom of practice.”

Review Unit I material on course portfolios

Review a course portfolio example.

The Course Portfolio

Where to Publish and Present

Venues for publication and presentation of work in SOTL abound. Journals and conferences may be discipline-based or inclusive of all disciplines, refereed or non-refereed, and local, regional, national, or international. Publications may be printed or on-line and conferences may be physical or virtual (teleconference).

Many sources of publication are evident in references given throughout the tutorial. Additionally, over 100 journals publishing SOTL are accessible through a list of journals publishing scholarship of teaching and learning.

Opportunities to present in cross-disciplinary venues (calls for proposals and conference announcements) are provided by the Professional and Organizational Development (POD) Network in Higher Education and the American Educational Research Association (Division J- Postsecondary Education). Both POD and AERA maintain electronic mail lists (listservers) which include calls for proposals and conference announcements from all over the world. More information about these lists are available at the above links.

http://www.indiana.edu/~deanfac/sotl/period.html

Potential Sources of External Funding

Pew Charitable Trusts

Fund for Improvement of Postsecondary Education

National Science Foundation

Woodrow Wilson National Fellowship Foundation

Alfred P. Sloan foundation

Department of Education

Spencer Foundation

Andrew W. Mellon Foundation

Eli Lilly Foundation

Sources of grant funding for specific topics and categories in higher education are generated by search engines operating on SPIN and COS databases. Most institutions subscribe to at least one of these databases. See your local sponsored research services or institutional research services for more information. Some sources, but by no means an exhaustive list, are shown below.

Task H: Reflecting on Classroom Research

Which major step in the classroom research schematic (determining the goal or issue, gathering essential information, or determining effect) seems the easiest for you to carry out? Why?

Which of the texts described in this section seems most immediately useful to you?

mailto:

Unit 3BQuestions, Designs, and Methods

•Framing Questions

•Task J: Examining Valerie’s Questions

•Task K: Framing Your Own Question

•Designs for Studies

•Choosing Measures

•Guiding Questions in Choosing Methodology

•Task L: Designing Your Own Project

•Summary of Standards

•Task M: Evaluation of the Tutorial

We begin the final module of the tutorial with a disclaimer; the possibilities are too many for adequate survey here. The topics in this module could easily fill a textbook (and, in fact, do fill many textbooks!) We limit coverage to a few possibilities.

Scholarship of teaching and learning does not necessarily begin with a question. To record “what worked,” the “wisdom of experience” or to describe observed phenomena without attempting intervention can constitute scholarship. To integrate the work of others, as in constructing a continuum of SOTL experience from Alpha to Omega, or as in editing a collection of papers into a book might also constitute scholarship. Nevertheless, much scholarship is rooted in inquiry because research is rooted in inquiry.

Faculty members do not necessarily receive formal preparation for teaching, much less scholarship of teaching. Therefore, to expect faculty members to know how to transform their issues and goals of teaching and learning into researchable questions without a little tutoring does not seem quite reasonable.

As many dissertation advisors and doctoral students will attest, the most difficult step in the whole research process is often framing the question to be addressed. A well framed question is one for which procedures can be devised that offer the possibility of arriving at an answer.

The next series of frames are rather narrowly focused on this particular challenge.

Framing the Question

1. Define a goal

2. Ask and answer questions to refine the goal (Colleagues are usually helpful.)

3. Create a succinct summary of a specified goal

4. Ask “what evidence would reveal that the goal is achieved?”

5. Frame possible researchable questions

Teachers commonly think of goals in terms of improved learning outcomes for students. Often, a study is built upon one or more interventions to achieve a goal and the question becomes “Does the intervention lead to achievement of the goal?” The goal approach also includes inquiries which Pat Hutchings might characterize as “What works?” However, a study based on a goal approach need not involve an intervention. A well-framed question could simply address whether an intended goal is presently being achieved or whether a goal is even feasible to achieve.

Framing Questions (Goal Approach)

Framing Questions (Issue Approach)

Criteria for selection of issues

• Investigable (not necessarily empirical)

• Bounded and well-defined

• Significant (not necessarily statistically)

Considerations for investigation of issues

• Length of time needed

• Complexity of procedures

• Availability of subjects

• Availability of support (resources, personnel, funds)

Questions can involve investigation of issues rather than achievement of goals; e.g. “How do students who do not meet prerequisites fare compared to those who do?

” In Opening Lines, Mills Kelly describes how his department chair framed Mills’ question by asking, “How do you know that using the Web as opposed to depending on paper (the way most of us have taught history) is transforming student learning – and, if so, whether for good or for ill?” Other types of questions like “What does the current learning environment for students in my course look like to them?” , “How do they perceive my instruction?” or “How do students who yield evidence of deep understanding in my course, gain that level of understanding?” can lead to investigative studies.

Making Vague Questions Answerable – Using Operational Definitions

Less framed

Do students who help others learn an academic discipline learn it better themselves?

Do students learn more in small classes?

What is the optimum number of homework assignments to give in a beginning math class?

More framed

Do students in CMSC 250 who tutor students in CMSC 150 perform better on the CMSC 250 final exam than students who do not tutor but have similar grades in CMSC 150?

Do students in sections of Phys 118 enrolling fewer than 50 students perform better on the departmental final exam than students from sections enrolling more than 75 students?

Do students enrolled in M036 who are given a homework assignment every week perform differently on the departmental final exam than students enrolled in M036 who are given homework every class period?

Vague questions are made answerable by using operational definitions of terms to improve conciseness. Operational definitions sharpen the question and afford possibility of clear answers. The price paid is usually a sacrifice in generality of answers obtained.

Considering small classes to be less than 50 students and “learning” to be characterized by final exam performance as in the above example are arguable definitions, of course. Nevertheless these operational definitions afford obvious and necessary clarity to the structure of the study.

Valerie Dean O’Loughlin Medical

Sciences


Review the introduction of Valerie’s project

Valerie’s Questions

1. Will the use of interactive learning activities in A215 lecture improve exam performance?

2. Will these activities improve lecture attendance and create a more active student engagement in lecture?

3. Will these activities improve overall “merit success rate” in A215?

Task J: Examining Valerie’s Questions

Valerie’s questions need further specification. What terms , in particular, need improved operational definition?

After submitting your answers, compare them with the answers in the following frame.

http://www.indiana.edu/~deanfac/sotl/sotlppt/SOTL_010209_2.html


Task J (Answers)Terms particularly needing improved operational definition:

“interactive learning activities”

Primary intervention: use of one or more interactive learning activities per lecture session

Examples of Learning Activities*

•Memory Matrices

•Learning Exercises

•Sample Exam Questions

•“Muddiest Point”

“active student engagement”

Valerie defines this term orally (beginning at 1 minute: 40 seconds into the video clip)

“merit success rate”

merit success rate = total A,B,C’s

initial enrollment

* Angelo, T and KP Cross (1993). Classroom Assessment Techniques: A Handbook for College Teachers. 2nd Ed. San Francisco: Jossey-Bass

Framing Questions as Hypotheses

H1: Students who write one-minute papers will perform better on a subsequent a. essay b. multiple choice quiz than students who do not write one-minute papers.

H2: Students who address their one-minute papers to a novice audience will perform better on a subsequent quiz than students who address their papers to the instructor.

H3: Students whose one-minute papers are graded will perform better on a subsequent quiz than students whose one-minute papers are not graded.

H4: The difference in quiz performance between students who do and do not write one minute papers will be less for higher ability students than for lower ability students.

Almer, E.D., Jones, K., & Moeckel, C.L. (1998) “The Impact of One-Minute Papers on Learning in an Introductory Accounting Course.” Issues in Accounting Education (13) 3

H1 actually contains two hypotheses. Why?

In some studies, questions are most usefully framed as hypotheses to be tested. All of these hypotheses were successfully tested in the referenced study.

Task K: Framing Your Question

1. Write a tentative question to be addressed in one of the courses you teach.

2. Discuss your question with one or two colleagues for the purpose of framing it in the clearest and most answerable way. Encourage your colleagues to challenge your framing.

3. Write your well-framed question at the end of this process.

THIS TASK MAY TAKE YOU AWHILE. You might want to stop here, think about your question, take it down the hall for discussion with colleagues, and bring back the results after lunch. You might also want to compare and contrast it to the questions in Faculty SOTL Projects before writing it in well-framed form and pressing the button to send it.

mailto:

What is a Design for a Study?

• A plan or protocol for carrying out the study

• An underlying scheme that governs functioning, developing, or unfolding

The design should include a timetable for significant milestones, specification of measures to be used in acquiring information, and provision for obtaining needed resources (Example of milestone: institutional approval to use your students as subjects. Two examples of needed resources: grade distributions of students in the target course over the last several years; a colleague to conduct interviews of your students.)

It may be necessary to modify the design as work progresses. In general, the greater the degree of qualitative information gathering, the more likely is modification in the design as the study progresses. It’s important that any design modification be the result of careful consideration rather than something that just happens in the course of events without the implications for the goals of the project having been thought through.

A good design always promotes efficient and successful gathering and analysis of the needed information.

Tthe design makes clear what information is being gathered (Example: student perception by means of fixed interview procedure in focus groups) or is being measured (Example: student learning by means of 3 exams). The design also makes clear what, if anything, is being manipulated. (Example: the study is designed so that some students attend review sessions before each exam while others do not.)

The nature and frequency of measurement and manipulation are made clear along with factors that the investigator is attempting to take into account because they may explain the result even though they are not the focus of the study. (Examples of such factors are students’ class attendance or backgrounds.)

A Qualitative or a Quantitative Study?

Danger: This may not be the best question to ask!

Why may the above question not be the best one to ask? Because it often tends to polarize faculty along their disciplinary lines and this may not be in the best interest of excellent scholarship of teaching and learning!

No factor is more influential in shaping our teaching and research perspectives than our individual disciplines. For most of us, almost our every word and thought about either teaching or research is grounded in our experience in the discipline. The research methods traditional in a discipline tend to influence the approaches faculty in that discipline take toward SOTL (and what journals in that discipline will accept for publication as SOTL).

An interesting experiment is to pose an issue for investigation to a cross-disciplinary group of faculty members. If encouraged to talk about approaches to the investigation, their approaches might be very different, reflecting their different disciplinary homes. Often each approach has potential to shed light on the issue. The scholarship that is most effective in advancing teaching and learning may be that which combines a variety of approaches.

But there is a danger. Rather than achieving synergy between different approaches, we sometimes tend to discredit or reject approaches which are not familiar to us.

I felt I was moving between two groups … who had almost ceased to communicate at all, who in intellectual, moral and psychological climate had so little in common … literary intellectuals at one pole – at the other scientists …Between the two a gulf of mutual incomprehension – sometimes hostility and dislike, but most of all lack of understanding. They have a curious distorted image of each other. Their attitudes are so different that even on the level of emotion, they can’t find much common ground. C. P. Snow*

*Snow, C. P. (1959) The Two Cultures and the Scientific Revolution. Cambridge University Press pp.2-4

Quantitative and Qualitative Methods as “Points of Possibility”

Quantitative

empirical, statistical, comparative

hypothesis testing, confirmatory

predetermined, fixed large, representative

scores, percentages, counts, rates

outsider, non-perturbing

deductive

summative, precise, reliable

Qualitative

naturalistic, fieldwork, constructivist

descriptive, generative, finding meaning

flexible, evolving small, purposeful

interviews, observations, writings

insider, perturbing

inductive

formative, rich, expansive

Feature

nature

goal

design

sample

measures

investigator

analysis method

findings

As in previous comparisons, we do not “go to the wall” in defense of the absolute correctness or comprehensiveness of the above table entries. Nevertheless, they are offered as generally useful characterizations.

Typical Measures Associated with Quantitative and Qualitative Methods

Qualitative

•performances (possibly recorded on tape)

•interviews (possibly recorded on tape)

•focus groups

•student projects, term papers, essay items or exams

•reflective statements

•journals

•reports of others (counselors, etc.)

Quantitative

•course exam, project, paper scores

•survey scores (Likert)

•frequencies of multiple choice test item responses

•scores on standardized scales and tests

•counts (participation, web requests, office visits)

•measures of time use

•institutional research data (GPAs, grades, admissions scores, frequency distributions of demographics)

In general, the more ways of gathering information and the more frequent the events in which information is gathered, the greater the validity of the study.

Assessment Measures

• “Affective” measures

– muddiest point, mid-course survey

• “Process” measures

– lecture attendance, Web hits/requests on learning activities

• “Performance” measures (for both the Class and the Instructor)

– Fall 2000 compared to Fall 99, 98, 97 semesters


Examples of Measures in a Particular SOTL ProjectReview the introduction to Valerie O’Loughlin’s project

Review Valerie’s research questions

Valerie discusses content of the box below

In the next 6 frames of the tutorial, some specific measures in Valerie’s study and the information gained from them is surveyed.

Skip over the examples of measures from Valerie’s study.

Examples of Measures in a Particular SOTL

Project: Mid-Semester Evaluation

– Lecture notes on Web very helpful

– More learning activities requested

– Extra lecture review sessions requested

– Review sheets requested

– Wanted more exams that cover less material


The mid-semester evaluation was a questionnaire designed by Valerie that students completed individually or in groups during class time and submitted anonymously. Key findings were:

The second finding confirmed that students liked the learning activities (which were the primary intervention in the study) and wanted more of them. Valerie granted the requests in 2, 3, and 4, during the semester of her study and is considering 5 for future semesters.

Examples of Measures in a Particular SOTL Project: Count of Web Hits on Learning Activities

0

2000

4000

6000

8000

10000

12000

14000

16000

# of page requests

Sept. Oct. Nov. Dec.

2000 1999 1998


Are students really accessing the learning activities?

Lecture Requests:

-Integument: 315

-Myology: 312

-Muscles of Upper Limb: 311

-Nervous System: 315

Learning Activity Requests:

-Epithelium: 237

-Myology 1: 159

-Muscles of Upper Limb: 155

-Neurons: 136

Average Number of web requests per student:

Fall 1998: 116 Fall 2000: 176

Web hits were counted as a measure of student use of learning activities outside of class. The left and right columns above respectively show number of requests for lecture notes and the associated learning activity during a particular month (September). The data below show that overall web hits associated with the target course increased sharply in the semester in which the learning activities were implemented.

Examples of Measures in a Particular SOTL Project: Pre-course data comparisons

SAT-V SAT-M GPA Credit Hours

1997 524 534 2.98 47

1998 533 545 3.04 48

1999 518 534 2.95 46

2000 516 536 2.95 49

Grand Avg.

524 537 2.99 47


The purpose of background measures is to investigate whether students across semesters are “fairly similar.” This controls for the possibility that any improvement shown on outcome measures in the study is because the class using the intervention consisted of higher achievers than the comparison group. In this case, the higher achievers appeared to be in the 1998 class rather than the class involved in the intervention (2000).


Project: Common Exam Item

0

10

20

30

40

50

60

70

80

% correct

2000 1999 1998 1997


• Motor neurons are examples of ______ neurons.

a. Bipolar

b. Afferent

c. Association

d. Pseudounipolar

e. Multipolar*

A common exam item can serve as a measure of learning. Except for minor changes in distractors, this item was the same over four years of tests in the target course. In the SOTL project (2000), a learning activity about types of neurons was done in class.


Project: Mean Exam Performance

Exam 1 Exam 2 Exam 3 Exam 4

2000 80.3 75.3 79.6 81.2

1999 74.4 72.5 76.5 76.9

1998 71.6 71.8 78 82.4

1997 74.7 69.8 75 78.3


Exam scores are frequently used as measures of learning. For a comparison of exam scores across years to have any meaning, the comparability of the exams across the years must be demonstrable. The comparability of exams was demonstrable in this study and students scored higher on 3 of 4 exams during the semester (Fall 2000) in which the learning activities were employed.

Examples of Measures in a Particular SOTL Project: Instructor Evaluations

• “The format of lectures, notes and learning exercises appealed to my style of learning”

• “She made learning easier by using different teaching techniques.”

• “She gave more personal attention in a class of 250+ than many do with much smaller classes. She consistently made us feel that she wanted us to succeed and that she would go the extra distance to make that happen.”

• “Dr. O’Loughlin is an excellent teacher. I speak as a humanities student who might turn to science if every science instructor were like her.”


Fall 2000 notables:

-highest overall mean evaluation

-highest score for question “my instructor clears up points of confusion for me”

-highest score for question “my instructor facilitates discussion among students”

3.73.83.9

44.14.24.34.44.54.64.74.8

1997 1998 1999 2000

The instructor evaluation is commonly used as both a quantitative and a qualitative measure. An end-of-course evaluation alone may not make a very compelling body of evidence in a study. However, when the evaluation is used in conjunction with a variety of quantitative and qualitative measures as in this SOTL project, a clear and compelling work of scholarship may result.

Guiding Questions in Choosing MethodologyWhat approach fits your research problem?

• Qualitative case study

• Quantitative study enhanced by qualitative data

• Qualitative study enhanced by quantitative data

A good primer: User Friendly Handbook for Mixed-Method Evaluations (NSF) Why such a publication from the premier organization representing the sciences? “ Because of the recognition that by focusing primarily on quantitative techniques, evaluators may miss important parts of a story.” Similarly by focusing exclusively on qualitative methods, researchers may miss opportunities to back their findings with the kind of objective evidence that makes the findings credible to a broader audience.

Guiding questions provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University

Do you have the skills/resources to carry out the methods?The skills/resources needed to employ qualitative and quantitative methods appropriately tend to be very different. Pooling of skills and resources is a good reason for community and collaboration in a campus SOTL initiative.

Will your audience find these approaches acceptable?

Editors or audiences within a discipline may be disinclined to accept work that does not include methods familiar to that discipline.

http://www.nsf.gov/cgi-bin/getpub?nsf97153

Task L: Designing Your SOTL Project

1. Individually design a study to address the question you framed in Task K.

2. Discuss your tentative design with with colleagues (perhaps the same ones who helped refine your question.) Encourage your colleagues to question you and comment.

3. Sketch any revisions to your design at the end of this process.

Like Task K, This task may take awhile to complete. Your design may involve sketches, or other graphics and tables or charts. Completing this task is an important step in processing the content of this unit. It is also a critical step if you plan to undertake a SOTL project!

mailto:

Summary of Standards

Clear GoalsDoes the scholar state the basic purpose of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?

Adequate PreparationDoes the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?

Appropriate MethodsDoes the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?

Significant Results

Does the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?

Effective Presentation

Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?

Reflective Critique

Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?

Glassick, C.E., Huber, M.T., Maeroff, G.I,. 1997 Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass.

The six general standards for scholarship below seem an excellent wrap-up of our work in the tutorial. George Walker comments on standards for SOTL

Task M: Closing Evaluation of Tutorial

1. My grasp of SOTL from Alpha to Omega is improved.

2. The tutorial had fresh significant perspectives of value.

3. I gained valuable resources for future use.

4. The tutorial was worth the time I spent working through it.

SD D U A SA

Please complete the above evaluation and press the button to e-mail responses. For example, if you strongly disagree with item 1, please reply 1SD, etc. Your comments are also most earnestly solicited. Thanks very much.

Documents

Almost every academician has heard the term scholarship of teaching. However, exchanges among colleagues from many institutions reveal a heterogeneity