Upload
kin
View
64
Download
0
Embed Size (px)
DESCRIPTION
Assessing Student Learning Outcomes in Student Development – Part II. Student Development Division Retreat SUNY Oneonta May 27, 2008. Presenter. Patty Francis, Associate Provost for Institutional Assessment and Effectiveness. Topics/Activities for Today. - PowerPoint PPT Presentation
Citation preview
Assessing Student Learning Outcomes in Student Development – Part II
Student Development Division RetreatSUNY OneontaMay 27, 2008
Presenter
Patty Francis, Associate Provost for Institutional Assessment and Effectiveness
Topics/Activities for Today
Continued themes from May 9 division meeting
Developing a unit assessment plan Writing good outcome statements Measuring student learning outcomes Hands-on opportunity to develop outcome
statements/measures for department
Continued Themes from May 9 Meeting
Assessment Basics and Rationale
Assessment as an ongoing, iterative process which must culminate with “closing the loop”
Importance of congruence in the assessment process
Assessment as an opportunity for ongoing dialogue, professional development and, most important, improving student services
Developing a Unit Assessment Plan
Overall, Unit Objectives Would Reflect:
Institutional effectiveness performance indicators
Documentation of all services and programs offered
Tracking of use of services (and by whom) Student satisfaction with services/programs Direct impact of services/programs on
students
Writing Good OutcomeStatements
Issues to Address at the Outset
Consistency between college, division, and unit mission statement (and, ultimately, outcome statements)
Do all staff have the opportunity to provide input?
Who are all your constituents? What results do you expect your
programs and services to produce?
Components of a Good Outcome Statement
Who is the target? What domain of student development is
the target? What change is expected?
But Other Things to Keep in Mind
Do you have a program/activity in place to bring about outcome?
Can desired change in students be measured?
How will you know you were successful? Do external standards apply (i.e., in case
of external accreditation/certification)?
Outcome Writing Rule #1: Focus on Student, Not Program
Good example: “Student workers will identify, provide, and implement technical equipment that is appropriate for specific union events.”
Poor example: “College union work study program will teach student workers how to select and set up equipment for union events.”
Outcome Writing Rule #2: State Outcomes Using Concrete Language and Action Verbs
Good example: “Students will negotiate necessary academic accommodations with faculty members.”
Poor example: “Our objective is to enhance students’ independence and self-confidence.”
Outcome Writing Rule #3: Focus on Results, Not Process
Good example: “Students will demonstrate increased job search skills (e.g., letter and resume writing, interviewing, employer research).”
Poor example: “Students will participate in three 2-hour sessions on letter writing, interviewing, and employer research.”
Measuring Student Learning Outcomes
Basic Principles Measurement techniques must be rigorous,
since unreliable data are of minimal value Best to use variety of quantitative and
qualitative measuresQuantitative easier, but not often as richQualitative often more informative, but require
check on scoring (e.g., rubrics) Rely as much as possible on existing data
sources
Types of Information to Include
1. Normative/benchmark information whenever possible (external sources, own performance over time)
2. Locally collected data
Survey data SUNY SOS NSSE Local surveys of
student satisfaction/ perceptions
National benchmarking data (e.g., ACUHO/EBI, ACUI/EBI)
Performance-based data Focus groups
General Types of Measures (Maki, 2004)
Direct measures – students actually demonstrate learning so that evaluators can match results to expectationsStandardized testsAuthentic, performance-based – embedded
into students’ actual educational context Indirect – students’ perceptions of learning
Should never be used as sole indicator
From Measures to Criteria Assessment criteria reflect your
expectations about student performance (i.e., how you know you were successful)
Set criteria at reasonable but challenging levels
Often take these forms: “90% of students will …..”“80% of students will score at least 70% on…”
Authentic, Performance-Based Assessment: The Value of Rubrics
Rubrics provide reliable means of rating student performance in more qualitative way
Steps in developing rubricsUse entire staff to help develop and be as
specific as possible in differentiating between performance levels
Use existing rubrics as guidePilot test to assure scoring is reliable
Rubric Example #1: Recreation & Sport Services (Univ. of W. Florida)
Variable Fails to Meet Approaches Meets ExceedsRespect Makes negative comments
about others and recreation. Engages in disrespectful behavior during work and/or participation.
Keeps self from making negative comments about others and recreation. Sometimes engages in disrespectful behavior during work and/or participation activities.
Sometimes makes positive comments about others and recreation. Generally engages in positive behavior during work and/or participation opportunities. May engage in disrespectful behavior during work and/or participation activities but learns from the experience and corrects behavior.
Frequently makes positive comments about others and recreation. Generally engages in positive behavior during work and/or participation opportunities. Rarely engages in disrespectful behavior during work and/or participation activities and always learns from the experience and corrects behavior.
Rubric Example #2: Career Services (Interviewing)
(Univ. of W. Florida)Variable Fails to Meet Approaches Meets ExceedsAppearance Did not dress in proper
interview attire or was messy and wrinkled - wore shorts, flip flops, hat, t-shirt, or other apparel that was improper
Wore casual khakis or jeans with a semi-casual shirt or blouse. Candidate may have not worn tie or professional heels
Professional appearance, but had one or two items that stood out to be less professional - for example, too short of skirt
Wore exceptional interview attire - conservative suit, proper length skirt, well-pressed garments and overall - well groomed
First Impression
May have been late to the interview, appeared too casual or uninterested initially, provided poor handshake or greeting, etc.
Was on time to the interview, but offered poor greeting, handshake or appearance, etc.
Was on time to the interview; looked interested and had a professional appearance - may need to practice greeting or handshake
Was on time to the interview, offered a professional appearance, and gave a great handshake and greeting
Non-Verbal Behavior
May have used many distracting hand gestures or no eye contact - Candidate appeared completely uninterested or heavily slouched
May have used distracting hand gestures or may have slouched in the interview for approximately half of their interview time
Used some distracting hand gestures or may have slouched some of the time during the interview, but overall looked interested
Used appropriate hand gestures and displayed good posture & appropriate eye contact - generally illustrated that they were interested
Communi-cation
May have organized thoughts in an unclear way, used slang, or may have been too concise or brief with answers given
May need some work with organizing thoughts and using examples to strengthen issues discussed
Organizes thoughts well, uses proper grammar, but may still need to use examples
Gives solid introductions and conclusions to thoughts, expresses self in proper grammar, and uses examples
Congruence in the Assessment Process: A Detailed Example
Handout #1
Your Turn!
Handout #2