View
215
Download
1
Category
Tags:
Preview:
Citation preview
Tutorial 5
Service-Learning Assessment
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Tutorial GoalsAt the end of this tutorial, you will have an understanding of:
Learn about the importance of assessing service-learning
Understand the assessment process
Realize the various strategies and rubrics used in assessment
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Tutorial Objectives
At the end of this tutorial you will be able
to:Generate potential assessment measures for
your service-learning course
Create potential rubric for assessment
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
What is Assessment?Method of gathering & interpreting information to identify whether we are successful at meeting student needs & achieving learning outcomes
Provides feedback to enable faculty to identify changes and make improvements in courses
A process not an event
Allows us to respond to the needs of students, community partners, and ourselves
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Why is Assessment Important? Improves the quality of academic programs Enhances student learning Makes explicit processes and outcomes that are
often implicit Can help faculty become better teachers Helps us provide more effective accounting of our
value to our communities and public constituencies
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Why is Assessment Important? Provides for wiser planning, budgeting, curriculum
change, staffing, programming, and student support, among others…
Creates a climate of caring and engagement that supports students’ own commitment to their learning
Helps us to be accountable for what we say we are doing
**You may have been doing assessment all along; you may just not have thought of it as assessment…
(Diamond, R. M. 1998; Walvoord, B. E., 2004) Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Partners for AssessmentComplete assessment should include all partners of the service-learning experience:
Student Assessment Instructor Assessment Institutional Assessment (may relate to program,
department…) Community Assessment
This tutorial will focus mainly on student-related assessment but ideas can easily be modified
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Steps for Effective Assessment
Begin with Clear Goals and Objectives Plan Design of Evaluation Select Data Collection Methods Collect Data Process and Analyze/Evaluate the Results Report Feedback from Results Make Appropriate Adjustments Based Upon
Feedback; loop starts all over again
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Goals and Objectives
Begin with your clear goals and objectives of what you want the students to learn and what you want them to be able to do at the end of the project/semester
Hopefully you already have thought of some goals/objectives in your initial planning of your service-learning project
Goals and objectives are also covered in tutorial #2 should you like to review those items
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Design: Two Main Types of Assessment Formative Assessment
Assessment conducted during the activity/project On-going as the activity/semester is taking place Provides feedback that can be used to modify, shape,
and improve as it is happening
Summative Assessment Conducted after the program/semester…is completed or
has been in operation for a length of time Helps to make judgments of what went well, what needs
to be improved for the next time Useful for making adaptations for a future activities Sometimes used as a before-after instrumentOffice of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Two Main Methods of Data Collection
Quantitative methods: Quantitative methods are for quantifying numbers and percentages; that is, for measurement. This can be effective for some evaluation.
Questions like: Was there a change? How big was the change? What was the satisfaction level?What proportion of students achieved the
objective?
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Methods The primary tools for quantitative data
collection are: Surveys - self administered (paper questionnaires) Personal interviews (face-to-face or phone) Tests Other objective measures of outcomes (perhaps
grades – careful with clearances and reporting though)
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Methods Qualitative methods: Qualitative methods are
best for gaining a deep understanding of students’ experiences, motivation, opinions, concerns, and so on…
Questions like: How have students civic behaviors changed based
upon their experiences? How have students’ perceptions of the elderly been
shaped by their community visits? How have the community members been impacted by
students’ presence in their homes?
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Methods The primary tools for qualitative data collection
are: Focus groups (moderated group discussions) Critical incident review Reflections/journal reading Open-ended questionnaires Personal interviews (one-on-one interactive
discussions) Observation (in-class or field observations are used
for capturing information on what is actually done and generating a qualitative sense of the experience)
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Creating Rubrics
Many times, rubrics are used to measure students’ behaviors, activities, performances…
Once you know your methods and what you want for data, you can create rubrics
(Gelman, S., B. Holland, et. al. 2001)
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Creating RubricsDetermine what you want to measure – more than likely an outcome, may be via skill, knowledge, or attitude
Do you simply want to divide students into two or three groupings, based on whether they have attained or exceeded the standard for an outcome? If so, then a short scale may be adequate
Do you want to use the rubric as a grading scale also? Then perhaps a 5-pt scale is more appropriate
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Creating Rubrics
Think of the best acceptable response for
this item (1-5 scale) – your 5Think of least acceptable – your 1Think of minimal/intermediate acceptable
response - your 3Develop descriptions and labels for each
item
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Sample of Major Assessment Competencies MeasuresAdapted from National Communication Association criteria, http://www.natcom.org/nca/Template2.asp
Criteria Exemplary-3 Satisfactory - 2 Unacceptable-1
Content Even, balanced information clearly andeffectively supports acentral purpose or thesisand displays a thoughtful, in-depth analysis of a sufficientlylimited topic. Readergains insight.
Topic is appropriate and is focused; provides support fora thesis and displaysevidence of a basic analysis. Reader gains some insight.
Paper does notsuccessfully identifythesis. Analysis is vagueor not evident. Reader isconfused or may be misinformed.
Writing Conventions
The writing is free, oralmost free, of errors ingrammar and spelling.
There are occasionalviolations in the writing, butthey don't represent a majordistraction or obscure meaning.
Errors are so numerousthat they obscure themeaning of the passage. The reader is confused and stops reading.
Rubrics can be used to:Measure performance on project or assignment with items linked to learning objectives
Ex; looking at linking concepts from class in reflection; analyzing societal implications of health care policies on the elderly via community experiences
Self-assessment done by students to gauge their self-perception of some aspect of course/outcome
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Process and Analyze Data May need some statistical knowledge; or
general descriptive analysis May be looking for relevant themes May be able to go a quick cross-check to see
whether objectives have been met Determined by data collection
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Report Feedback from Results
With your guidance and comments, assessment can provide students with feedback on their performance and direction for improvement
Helps instructors better articulate the effectiveness of this approach to learning; Are students understanding material? Are they successful with achieving outcomes? Are you effective at getting ideas across to your students?
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Report Feedback from Results Keep careful track of your results so you can report
and accumulate feedback on an ongoing process
Always keep in mind that feedback from assessment can be an advantage in the future (i.e., funding purposes, obtaining resources, demonstration of teaching effectiveness for tenure and promotion…) in addition to the basic intention of knowing whether learning outcomes are achievedOffice of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Make Appropriate Adjustments Based Upon Feedback
Once you have analyzed the results, you need to close the feedback loop with making any adjustments in your assessment plan
Do your results give you the information to link directly to your outcomes? Can you determine from your instrument whether students are meeting the outcome? If not, perhaps the assessment instrument needs to be modified
Do your results tell you that students are not meeting the intended outcome you want? Different issue---perhaps you need to modify the outcome
Once you make adjustments, begin the process again to see if your adjustments worked and continue assessing…
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
General Tips for Assessment Do not try to re-invent the wheel
Look for those instruments and rubrics already created that you can modify for your purposes
Create a plan and try to stick to it Always different/perhaps better ways to do assessment Save that for the feedback look and adjustment step
Don’t be afraid to ask for help You may not know everything about assessment Tap into your resources on campus, websites, others
texts… Assume that assessment is “messy”
Be sensitive to people error and have a sense of humorOffice of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Assessment with Different Parties of Service-Learning
We covered mainly student assessment; other issues to consider with faculty, community partners and institutional/programmatic assessment
Basically same principles, but perhaps different data collection and issues to consider with providing results and feedback; different purposes
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Final Thoughts Learned the steps for conducting assessment Covered some different types of assessment
techniques and methods of data collection for examining your learning outcomes
Learned how to create rubrics, which can be helpful in various teaching & assessment purposes
Hopefully you are well on your way to be confident in your service-learning classroom and will be ready to determine whether your students are really benefiting from their community experiences
Various assessment samples and grids for you to example and complete on the web pages
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
References
Diamond, R. M. (1998). Designing & assessing courses & curricula: A
practical guide. San Francisco: Jossey-Bass.
Gelman, S., B. Holland, et. al. 2001. Assessing Service-Learning and Civic Engagement:Principles and techniques. Providence, RI: Campus Compact.
Howard, J. (Ed.) (2001). Service-learning course design workbook. Michigan Journal of
Community Service Learning. Ann Arbor, Mi: OCSL Press, The University of
Michigan
Walvoord, B. E., (2004). Assessment clear and simple: A practical guide for
institutions, departments, and general education. San Francisco:
Jossey-Bass
http://www.natcom.org/nca/Template2.asp
Office of Academic Excellence and Assessment Creighton University
© 2007 Donna R. Pawlowski, Ph.D. Associate Professor
Department of Communication Studies
Recommended