Tutorial 5 Service-Learning Assessment Office of Academic Excellence and Assessment Creighton University © 2007 Donna R. Pawlowski, Ph.D. Associate Professor

Tutorial 5

Service-Learning Assessment

Office of Academic Excellence and Assessment Creighton University

© 2007 Donna R. Pawlowski, Ph.D. Associate Professor

Department of Communication Studies

Tutorial GoalsAt the end of this tutorial, you will have an understanding of:

Learn about the importance of assessing service-learning

Understand the assessment process

Realize the various strategies and rubrics used in assessment




Tutorial Objectives

At the end of this tutorial you will be able

to:Generate potential assessment measures for

your service-learning course

Create potential rubric for assessment




What is Assessment?Method of gathering & interpreting information to identify whether we are successful at meeting student needs & achieving learning outcomes

Provides feedback to enable faculty to identify changes and make improvements in courses

A process not an event

Allows us to respond to the needs of students, community partners, and ourselves




Why is Assessment Important? Improves the quality of academic programs Enhances student learning Makes explicit processes and outcomes that are

often implicit Can help faculty become better teachers Helps us provide more effective accounting of our

value to our communities and public constituencies




Why is Assessment Important? Provides for wiser planning, budgeting, curriculum

change, staffing, programming, and student support, among others…

Creates a climate of caring and engagement that supports students’ own commitment to their learning

Helps us to be accountable for what we say we are doing

**You may have been doing assessment all along; you may just not have thought of it as assessment…

(Diamond, R. M. 1998; Walvoord, B. E., 2004) Office of Academic Excellence and Assessment Creighton University



Partners for AssessmentComplete assessment should include all partners of the service-learning experience:

Student Assessment Instructor Assessment Institutional Assessment (may relate to program,

department…) Community Assessment

This tutorial will focus mainly on student-related assessment but ideas can easily be modified




Steps for Effective Assessment

Begin with Clear Goals and Objectives Plan Design of Evaluation Select Data Collection Methods Collect Data Process and Analyze/Evaluate the Results Report Feedback from Results Make Appropriate Adjustments Based Upon

Feedback; loop starts all over again




Goals and Objectives

Begin with your clear goals and objectives of what you want the students to learn and what you want them to be able to do at the end of the project/semester

Hopefully you already have thought of some goals/objectives in your initial planning of your service-learning project

Goals and objectives are also covered in tutorial #2 should you like to review those items




Design: Two Main Types of Assessment Formative Assessment

Assessment conducted during the activity/project On-going as the activity/semester is taking place Provides feedback that can be used to modify, shape,

and improve as it is happening

Summative Assessment Conducted after the program/semester…is completed or

has been in operation for a length of time Helps to make judgments of what went well, what needs

to be improved for the next time Useful for making adaptations for a future activities Sometimes used as a before-after instrumentOffice of Academic Excellence and Assessment Creighton University



Two Main Methods of Data Collection

Quantitative methods: Quantitative methods are for quantifying numbers and percentages; that is, for measurement. This can be effective for some evaluation.

Questions like: Was there a change? How big was the change? What was the satisfaction level?What proportion of students achieved the

objective?




Methods The primary tools for quantitative data

collection are: Surveys - self administered (paper questionnaires) Personal interviews (face-to-face or phone) Tests Other objective measures of outcomes (perhaps

grades – careful with clearances and reporting though)




Methods Qualitative methods: Qualitative methods are

best for gaining a deep understanding of students’ experiences, motivation, opinions, concerns, and so on…

Questions like: How have students civic behaviors changed based

upon their experiences? How have students’ perceptions of the elderly been

shaped by their community visits? How have the community members been impacted by

students’ presence in their homes?




Methods The primary tools for qualitative data collection

are: Focus groups (moderated group discussions) Critical incident review Reflections/journal reading Open-ended questionnaires Personal interviews (one-on-one interactive

discussions) Observation (in-class or field observations are used

for capturing information on what is actually done and generating a qualitative sense of the experience)




Creating Rubrics

Many times, rubrics are used to measure students’ behaviors, activities, performances…

Once you know your methods and what you want for data, you can create rubrics

(Gelman, S., B. Holland, et. al. 2001)




Creating RubricsDetermine what you want to measure – more than likely an outcome, may be via skill, knowledge, or attitude

Do you simply want to divide students into two or three groupings, based on whether they have attained or exceeded the standard for an outcome? If so, then a short scale may be adequate

Do you want to use the rubric as a grading scale also? Then perhaps a 5-pt scale is more appropriate




Creating Rubrics

Think of the best acceptable response for

this item (1-5 scale) – your 5Think of least acceptable – your 1Think of minimal/intermediate acceptable

response - your 3Develop descriptions and labels for each

item







Sample of Major Assessment Competencies MeasuresAdapted from National Communication Association criteria, http://www.natcom.org/nca/Template2.asp

Criteria Exemplary-3 Satisfactory - 2 Unacceptable-1

Content Even, balanced information clearly andeffectively supports acentral purpose or thesisand displays a thoughtful, in-depth analysis of a sufficientlylimited topic. Readergains insight.

Topic is appropriate and is focused; provides support fora thesis and displaysevidence of a basic analysis. Reader gains some insight.

Paper does notsuccessfully identifythesis. Analysis is vagueor not evident. Reader isconfused or may be misinformed.

Writing Conventions

The writing is free, oralmost free, of errors ingrammar and spelling.

There are occasionalviolations in the writing, butthey don't represent a majordistraction or obscure meaning.

Errors are so numerousthat they obscure themeaning of the passage. The reader is confused and stops reading.

Rubrics can be used to:Measure performance on project or assignment with items linked to learning objectives

Ex; looking at linking concepts from class in reflection; analyzing societal implications of health care policies on the elderly via community experiences

Self-assessment done by students to gauge their self-perception of some aspect of course/outcome




Process and Analyze Data May need some statistical knowledge; or

general descriptive analysis May be looking for relevant themes May be able to go a quick cross-check to see

whether objectives have been met Determined by data collection




Report Feedback from Results

With your guidance and comments, assessment can provide students with feedback on their performance and direction for improvement

Helps instructors better articulate the effectiveness of this approach to learning; Are students understanding material? Are they successful with achieving outcomes? Are you effective at getting ideas across to your students?




Report Feedback from Results Keep careful track of your results so you can report

and accumulate feedback on an ongoing process

Always keep in mind that feedback from assessment can be an advantage in the future (i.e., funding purposes, obtaining resources, demonstration of teaching effectiveness for tenure and promotion…) in addition to the basic intention of knowing whether learning outcomes are achievedOffice of Academic Excellence and Assessment Creighton University



Make Appropriate Adjustments Based Upon Feedback

Once you have analyzed the results, you need to close the feedback loop with making any adjustments in your assessment plan

Do your results give you the information to link directly to your outcomes? Can you determine from your instrument whether students are meeting the outcome? If not, perhaps the assessment instrument needs to be modified

Do your results tell you that students are not meeting the intended outcome you want? Different issue---perhaps you need to modify the outcome

Once you make adjustments, begin the process again to see if your adjustments worked and continue assessing…




General Tips for Assessment Do not try to re-invent the wheel

Look for those instruments and rubrics already created that you can modify for your purposes

Create a plan and try to stick to it Always different/perhaps better ways to do assessment Save that for the feedback look and adjustment step

Don’t be afraid to ask for help You may not know everything about assessment Tap into your resources on campus, websites, others

texts… Assume that assessment is “messy”

Be sensitive to people error and have a sense of humorOffice of Academic Excellence and Assessment Creighton University



Assessment with Different Parties of Service-Learning

We covered mainly student assessment; other issues to consider with faculty, community partners and institutional/programmatic assessment

Basically same principles, but perhaps different data collection and issues to consider with providing results and feedback; different purposes




Final Thoughts Learned the steps for conducting assessment Covered some different types of assessment

techniques and methods of data collection for examining your learning outcomes

Learned how to create rubrics, which can be helpful in various teaching & assessment purposes

Hopefully you are well on your way to be confident in your service-learning classroom and will be ready to determine whether your students are really benefiting from their community experiences

Various assessment samples and grids for you to example and complete on the web pages




References

Diamond, R. M. (1998). Designing & assessing courses & curricula: A

practical guide. San Francisco: Jossey-Bass.

Gelman, S., B. Holland, et. al. 2001. Assessing Service-Learning and Civic Engagement:Principles and techniques. Providence, RI: Campus Compact.

Howard, J. (Ed.) (2001). Service-learning course design workbook. Michigan Journal of

Community Service Learning. Ann Arbor, Mi: OCSL Press, The University of

Michigan

Walvoord, B. E., (2004). Assessment clear and simple: A practical guide for

institutions, departments, and general education. San Francisco:

Jossey-Bass

http://www.natcom.org/nca/Template2.asp




Documents

Tutorial 5 Service-Learning Assessment Office of Academic Excellence and Assessment Creighton University © 2007 Donna R. Pawlowski, Ph.D. Associate Professor