USING RUBRICS TO ASSESS THE PERFORMANCE OF NEW

PROFESSIONALS IN STUDENT AFFAIRS

Tony Ribera & Sarah FernandezA Presentation at the 2009 NASPA Region IV-E Conference,

Chicago, IL

AGENDA

State of assessment in student affairs Integrating rubrics into assessment efforts New professionals in student affairs and

using rubrics to assess their learning/performance

Going forward

PARTICIPANTS SHOULD GAIN…

An appreciation for rubrics as a way to communicate expectations and assess performance

An understanding of steps to develop a descriptive rubric in various functional areas of student affairs

Experience creating a descriptive rubric with knowledgeable colleagues from various institutions

ASSESSMENT IN STUDENT AFFAIRS

Who is responsible (ACPA, 2007) Questioning student learning (Creamer,

Winston, & Miller, 2001; Hanson, 1991; King, 2003; Love, 1995)

Systematic inquiry (Pascarella & Whitt, 1999) Looking toward the future (Torres & Walbert,

2009)

“One can argue there is a large gap between what student affairs practitioners say about the value and importance of research and assessment to their work and the extent to which such activities are actually conducted and used” (Pascarella & Whitt, 1999, p. 103).

WHAT IS A RUBRIC?

Assessment tool often used to grade assignments

Divides a larger assignment into smaller tasks

Describes criteria and sets expectations Allows for consistent scoring Provides appropriate feedback

(Stevens & Levi, 2005; Suskie, 2009)

TYPES OF RUBRICS (Suskie, 2009)

Checklists List of goals/outcomes

Rating Scales List of goals/outcomes plus rating scale

Descriptive Rubrics Includes description for each level of the rating

scale Holistic Scoring Guides

Narrative description of the ratings possible. Does not include list of goals/outcomes

DESCRIPTIVE RUBRICS (Stevens & Levi, 2005)

Most commonly used in higher education Made up of 4 parts:

Task Description Description of overall assignment

Scale Provides labels for the rating of performance

Dimensions Goals/outcomes. Skills that should be achieved

Description of Dimensions Expectations for each dimension at each rating scale

ASSESSMENT AND NEW PROFESSIONALS

“Learning must be at the center of the student affairs profession. We as student affairs professionals require the consistent development of our own learning. Thus, what we learn and how we learn it becomes critical to developing our role as facilitators of the student learning and development process” (ACPA, 2007, p.2).

WHAT DO WE KNOW ABOUT NEW PROFESSIONALS?

Graduate preparation (Waple, 2006) Competencies and responsibilities (Burkard,

Cole, Ott, & Stoflet, 2005) Values in recruiting and hiring (Kretovics,

2002) Desire for feedback (Renn & Jessup-Anger,

2008)

CHALLENGES FACING NEW PROFESSIONALS

Graduate preparation (Erwin & Sivo, 2001) Time/lack of resources (Johnson & Steele,

1984) Individual interest and motivation

Exemplary Competent Basic

Application of Professional Literature in program and intervention development

Use literature to understand new assessment techniques. Contribute to literature on assessment and evaluation.

Use literature to understand new assessment techniques.

Use literature to understand the importance of assessment and evaluation techniques

Facilitation of data collection to measure student learning

Design and implement high quality assessment and evaluation strategies appropriate for staff, departmental , and institutional efforts. Facilitate training of staff to participate in assessment techniques.

Recognize importance of assessment, and participate in training of colleagues. Identify most appropriate instruments to be used in assessment efforts for staff and departmental efforts. Construct basic assessment and evaluation tools with supervision.

Collect data to assess department wide educational efforts.Evaluate staff, collecting data from multiple sources.

Interpretation of assessment data

Facilitate the development of assessment and evaluation reports, which use data to create goals and plans of action for use of results. Use departmental resources to support high quality evaluation.

Participate in the analysis of collected data.Effectively interpret results to be reported to department.

Interpret data in order to use results of reports and studies.

Application of results to improve programs and practices

Ensure the use of results in improving and/or creating new departmental and institutional efforts, programs, activities. Emphasize importance of assessment to departmental and institutional efforts.

Ensure the use of results in improving and/or creating new departmental efforts, programs, activities. Emphasize importance of assessment to departmental efforts.

Use findings of assessment and evaluation in improving educational efforts.

Comprehension of qualitative research methods

Supervise the design of qualitative assessment and evaluation. Understand needs to be flexible in qualitative studies. Have mature understanding of transferability and trustworthiness .

Create qualitative sampling designs with supervision. Participate in analysis of qualitative data.Use techniques to ensure trustworthiness.

Understand qualitative research methods. Understand how qualitative findings can be used in work settings. Judge trustworthiness of study.

Comprehension of quantitative research methods

Supervise the design of quantitative assessment and evaluation. Understand how to disseminate results. Assess instruments for validity and reliability. Learn to use mixed methods of data collection.

Create quantitative designs with supervision. Participate in the analysis of quantitative data. Understand more complex methods, including multivariate techniques.

Understand quantitative research methods. Recognize issues of validity and reliability in quantitative methods.

STEPS FOR CREATING A DESCRIPTIVE RUBRIC (Suskie, 2009)

Identify learning outcomes-what do you want them to take away?

List criteria-what do you want them to accomplish? Focus on most significant skills. These will

become your dimensions Developing rating scale

At minimum levels for adequate and inadequate Write descriptions Test it out- make sure the standards and

descriptions are appropriate

RUBRIC DEVELOPMENT MODELS (Stevens & Levi, 2005)

Presentation Model Rubric presented without outside feedback

Feedback Model Rubric can be changed with feedback after

presented Pass-the-Hat Model & Post-it Model

Both parties work together to develop rubric 4 X 4 model

Students take full responsibility of development

SMALL GROUP WORK ACTIVITY AND LARGE GROUP DISCUSSION

MOVING FORWARD

Need for more scholar-practitioners (Hossler, 2001; Komives, 1998; Schroeder & Pike, 2001)

Engaging in a scholarship of teaching and learning (Huber & Hutchings, 2005) Questioning Gathering and exploring evidence Trying out and refining new insights Going public