Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302

Anne Marie Karlberg Director of Institutional Research and Assessment

(360) 383-3302amkarlberg@whatcom.ctc.edu

http://faculty.whatcom.ctc.edu/InstResearch/index.htm

Assessment 101

PNAIRP Conference Thursday October 1, 2009

Outcomes

1. Define assessment.

2. Identify the components of a successful assessment program.

3. Articulate three types of assessment information or data and provide examples of each.

4. Identify three levels of assessment.

5. Identify two phases in creating an effective outcomes process.

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

What is assessment?

“ The systematic collection of information about student learning…to inform decisions about how to improve learning.”

(Walvoord, 2004, p. 2)

Purposes

1. improvement (formative)1. improvement (formative)

2. accountability (summative)

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

Educational Evaluation Model: CIPPBy Daniel Stufflebeam

Using CIPP to Create an Effective Assessment Program

Using CIPP to Create an Effective Assessment Program: Context

Context: What do we know about the context of our

colleges? At Whatcom Community College…

8.3% of our employees

18.9% of our degree and certificate seeking students

16.0% percent of Whatcom County residents

…indicate they are of color.

Using CIPP to Create an Effective Assessment Program: Inputs

Resources

• human resources 

• financial support 

• technical support 

• administrator and faculty support

Plans and Strategies

Whatcom will…

•revise its mission statement and familiarize faculty / staff with mission

•implement the strategic plan

•implement the assessment plan

Using CIPP to Create an Effective Assessment Program: Processes

Implementation

Link assessment program to mission statement • redirect resources towards priorities • increase responsiveness to community needs•measure the extent to which we are successful

Provide opportunities for meaningful, collaborative college-wide conversations, by structuring assessment• in an ongoing, simplified, participatory way • relevant and meaningful

Embed assessment throughout college

Learning and teaching practices

Apply meaningful, relevant and contextualized experiences for students, such as…

→ using self-reflection→ applying concepts to a relevant context→ teaching material to peers→ discovering connections between subjects

Using CIPP to Create an Effective Assessment Program: Products

1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations

Assessment data

1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations

2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews

Assessment data

1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations

2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews

3.Institutional data: e.g., retention, graduation, enrollment, transfer trends

Assessment data

A.College level

B.Program level

C.Course level

Assessment levels

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

3. Institutional data (rates and numbers)

Assessment Products: Examples

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

3. Institutional data (rates and numbers)

Assessment Products: Examples

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Using CIPP to Create an Effective Assessment Program

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products

Two phases of an outcomes process

(1) Development of an outcomes process

(2) Implementation of an outcomes process

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

Using CIPP to Create an Effective Assessment Program

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Direct indicators: College outcomes plan

Goal (# of CLAs)

Whatcom will…

Baseline (May 2008) 2008 - 2009

2009 - 2010

2010 - 2011

2011 - 2012

(1) Development of the college outcomes processa. educate faculty / staff / students about assessment No formal

initiativesongoing ongoing ongoing ongoing

b. revise Core Learning Abilities (CLAs) and outcomes Identified 5 CLAs in 1998

Revise CLAs and outcomes

refine review review

c. develop assessment tools (e.g., scoring guides / rubrics) to measure the outcomes

5 review review

d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)

5 review review

e. include outcomes on syllabi 1 2f. collect the instructional assignments, activities, projects, or experiences, in required courses that will be used to teach outcomes at entry, midway, and exit

1 2

g. collect the activities, experiences, projects, essays, or assignments in required courses that will be used to assess outcomes at entry and exit

1 2

h. attach anchor papers (i.e., examples) for each level of the scoring guide/rubric scale

1 2

(2) Implementation of the college outcomes processa. assess students at (entry and) exit for outcomes   1b. analyze the (entry and) exit assessment data   1c. present analysis to faculty and students and consult on the results     1d. use the data to improve and revise curriculum   1e. document the process; create an assessment report about how the data were used to improve learning

  1

Direct indicators: College outcomes plan

Goal (# of CLAs)

Whatcom will…

Baseline (May 2008)

2008 - 2009

2009 - 2010

2010 - 2011

2011 - 2012

(1) Development of the college outcomes processa. educate faculty / staff about assessment No formal

initiativesOngoing

b. revise Core Learning Abilities (CLAs) and outcomes

Identified 5 CLAs in

1998

Revise CLAs and outcomes

Refine Review

c. develop assessment tools (e.g., scoring guides / rubrics, etc.) to measure the outcomes

Review

d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)

Review

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Indirect indicators:College survey schedule

Goal

Survey

Baseline data (May 2008)

2008 – 2009

2009 – 2010

2010 – 2011

2011 – 2012

2012 – 2013

Student Opinion Survey (every 6 years – alternate with the CCSSE)

Conducted January 2008

Community College Survey of Student Engagement (every 6 years – alternate with the Student Opinion Survey)

Faculty Survey of Student Engagement (every 6 years)

Staff satisfaction survey (every 4 years) Conducted in 2007)

Alumni Survey (ACT with Whatcom-specific questions) (every 10 years)

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Institutional data: College-level plan

Type of Data

Baseline May 2008

2008 – 2009

2009 – 2010

2010 – 2011

2011 – 2012

Enrollment numbers

Profile of all students

Grade distribution

Graduation rates / numbers

Retention rates

Success after transfer to WWU

Overview

1. What is assessment?

2. Components of an effective assessment program

3. Phases of an outcomes process

4. Assessment plan

5. Website

Website

•WCC Homepage

• “About Whatcom”

• “Assessment / Inst Research”

http://www.faculty.whatcom.ctc.edu/InstResearch/

Comments and questions?

Using CIPP to Create an Effective Assessment Program

Type of data College Program Course

1. Direct indicators (outcomes assessment)

• Core Learning Abilities and outcomes

• Program outcomes

• Course outcomes

2. Indirect indicators (surveys, interviews)

•Student opinion survey

•CCSSE

•Graduate survey

•Alumni survey

•Student evaluation of courses

3. Institutional data (rates and numbers)

• Graduation • Performanc

e after transfer

•Student enrollment

•Retention •Transfer

•Course completion

•Grade distribution

Assessment Products: Examples

Anne Marie Karlberg (Director of Institutional Research and Assessment)(360) 383-3302

amkarlberg@whatcom.ctc.eduhttp://faculty.whatcom.ctc.edu/InstResearch/index.htm