Academic Program Review SELF-STUDY APPENDICES of... · Appendix 2 California State University, Stanislaus Substitution of Specialized Accreditation Self Study for the Academic Program

1

Academic Program Review

SELF-STUDY APPENDICES

Appendix 1 California State University, Stanislaus

Academic Program Review Signature Page

California State University, Stanislaus: Academic Program Review Self-Study Appendices 1

________________________________________________________________

Title of Program

Signatures:

_________________________________________________________________________________ _ ___________ Program Faculty Member (Print) Signature Title Date _________________________________________________________________________________ _ ___________ Program Faculty Member (Print) Signature Title Date

_________________________________________________________________________________ _ ___________ Program Faculty Member (Print) Signature Title Date

_________________________________________________________________________________ _ ___________ Program Faculty Member (Print) Signature Title Date _________________________________________________________________________________ _ ___________ Program Faculty Member (Print) Signature Title Date




_________________________________________________________________________________ _ ___________ Department Chair (Print) Signature Title Date

_________________________________________________________________________________ _ ___________ College Curriculum Committee (Print) Signature Title Date

_________________________________________________________________________________ _ ___________ General Education Subcommittee (Print) Signature Title Date

_________________________________________________________________________________ _ ___________ College Dean (Print) Signature Title Date

_________________________________________________________________________________ AVP (Academic Affairs) and ALO (Print) Signature Title Date

________________________________________________________________________________ Graduate Council (if applicable) (Print) Signature Title Date

________________________________________________________________________________ University Educational Policies Committee Signature Title Date (if applicable) (Print)

_________________________________________________________________________________ Provost and VP for Academic Affairs Signature Title Date


Substitution of Specialized Accreditation Self Study for the Academic Program Review Self Study


For programs that undergo professional or specialized accreditation, academic program review is coordinated with the accreditation or re-accreditation cycle. The self study developed for professional or specialized accreditation reviews, provides the essential requirements of academic program review and may be used for this purpose with approval by the college dean. The completed table below and the APR signature page (APR Procedures, Appendix 1) are to be forwarded by the college dean to the AVP/ALO for substitution of accreditation self study for the APR self study.

Program________________ Accreditation Agency__________________ Date of Self-Study_________________

CSU Stanislaus Academic Program Review Component Specialized Accreditation Standard

A. Pr

Program Overview

1. Introduction and Historical Context

2. Relationship to Mission (University and College) and Strategic Plan

3. Program Description (with course requirements)

4. Review of Changes since the Last APR/ Accreditation Review

Data Review 1. Student Profile and Enrollment Trends (demographics)

2. Faculty Profile

3. Delivery of Instructional Program

Location (Turlock, Stockton, Merced, etc.)

Distance Education (online, ITV)

Scheduling of classes (3 year plan)

Justification of additional units

4. Peer Institutions/Benchmarking

Commitment to Student Learning

1. Faculty

Advising and Mentoring

Teaching

Research, Scholarship, and Creative Activity

2. Assessment of Student Learning

Program Learning Outcomes/Student Learning Outcomes

Curriculum Map

Student Assessment

Student Achievement

Resources 1. Facilities (classroom space, equipment)

2. Fiscal (budget review)

3. Technology Resources and Support

4. Library Resources and Information Literacy

Implementation Plan

1. External Consultants/Accreditation Review Date of Review Visit_________________________ Date of Review Report_______________________ Date of Response to Review Report____________

2. Implementation Action Plan

Note: Appendix 1 (Academic Program Review Signature Page) to accompany Appendix 2.


Graduate Assessment – DRAFT


GRADUATE EDUCATION ASSESSMENT: FAQS

Why do we need to assess student learning in graduate programs?

1. Going through the process of developing an assessment plan ensures that your program curriculum is

logically designed and reflects the desired student learning outcomes.

2. Assessment data provides evidence that students are meeting faculty expectations for graduates of the

program.

3. Assessment is required by WASC, disciplinary accreditation agencies, and for grants such as CEGE, in order

to determine if students (or specific sub-groups of students) are meeting student learning outcomes.

What are the major steps for developing a graduate assessment plan (linked to APR Procedures)?

1. Program faculty decide on what they want students to learn (Section V-A).

2. Faculty identify the courses that emphasize the various learning outcomes (Section V-B).

3. Faculty identify one or more signature assignments that provide opportunities for students in the

program to show their mastery of learning outcomes (Section V-D). Examples of signature assignments

could include term papers, exams, projects, and theses.

(Note: Steps 1-3 are incorporated into the curriculum map- Section V-B)

4. Faculty develop rubrics for each signature assignment, to facilitate evaluation and comparison across

courses or cohorts of students. Ideally, each of the program learning outcomes are included in one or

more rubrics. (Section V-E)

5. Assignments are graded by course instructors (and/or additional reviewers). Students’ scores on signature

assignments become the raw data for the direct assessment of student learning outcomes (Section V-D).

6. Data from the direct assessment of student learning (with additional data from indirect assessments, e.g.

surveys) are used to evaluate the degree to which students are meeting program learning outcomes, and

trends over time. Data are reported in annual reports and used for academic program reviews (Section

IX).




GRADUATE EDUCATION: ASSESSMENT PLAN

Over a decade ago, the Graduate Council was a leader in creating an assessment approach centered on student

learning goals for graduate education and continues its commitment, as a collective governance body, to

promoting and evaluating graduate program quality. The Graduate Council created university‐wide graduate

student learning goals that link to the major discipline‐specific program learning outcomes unique to each

graduate program (drafted 1999; approved in 2000). Since that time, the Graduate School has employed various

assessment methods for collecting information that has assisted the Graduate Council in its consideration of the

quality of graduate programs (see Table 1). These methods collectively contribute to answering the important

question of the degree to which our graduate programs achieve their shared goal of educating graduates.

The assessment methods described in the Academic Program Review procedures are aligned with and

complement the assessment methods used by individual graduate programs. Most importantly, the assessment

strategies adhere to the University’s Principles for the Assessment of Student Learning (2004) which defines the

role of assessment within the institution.

The Graduate Council recognizes the complexity of assessment and the significance of designing methods that are

multidimensional, meaningful, and oriented toward program improvement and enhanced student learning.

Further, the Graduate Council subscribes to the philosophical conviction that the quality of teaching is inextricably

connected to the quality of student learning. Thus, while recognizing the importance of student learning outcomes

as an important component of program assessment, the Graduate Council avoids reliance on this measure alone as

it engages in a critical, comprehensive analysis of the quality of our graduate programs and our graduate students’

academic achievement.

Graduate Student Learning Goals

The six overall graduate student learning goals follow. Students will demonstrate…

1. advanced knowledge, skills, and values appropriate to the discipline.

2. ability to be creative, analytical, and critical thinkers.

3. ability to work as individual researchers/scholars as well as in collaboration with others in contributing to the

scholarship of their disciplines, as appropriate.

4. relevant knowledge of the global perspectives appropriate to the discipline.

5. knowledge of new and various methods and technologies as appropriate to the discipline.

6. advanced oral and written communication skills, complemented, as appropriate to the discipline, by the

ability to access and analyze information from a myriad of primary, print, and technological sources.

http://www.csustan.edu/oaqa/PrinciplesofAssessment.html




Table 1 displays the alignment between the graduate student learning goals and methods of assessment (both

direct and indirect methods).

Table 1

Alignment of Graduate Student Learning Goals and University-Wide Assessment Methods

Graduate Student Learning Goals

Assessment Methods

Aca

dem

ic P

rogr

am R

evie

w

Acc

red

itat

ion

*Ad

mis

sio

n E

xam

inat

ion

s

*Cu

lmin

atin

g Ex

per

ien

ce: T

hes

is,

Pro

ject

, Co

mp

Exa

m

Exte

rnal

Co

nsu

ltan

ts

Gra

de

Po

int

Ave

rage

*Stu

den

t Sc

ho

lars

hip

Stu

den

t A

war

ds

and

Ho

no

rs

Gra

du

ate

Sch

oo

l Exi

t Su

rvey

Gra

du

ate

Alu

mn

i Su

rvey

IDEA

Co

urs

e Ev

alu

atio

ns

Gra

du

ate

Nat

ion

al S

urv

ey o

f

Stu

den

t En

gage

me

nt

Gra

du

ate

Facu

lty

Surv

ey o

f

Stu

den

t En

gage

me

nt

Pro

gram

Ap

pro

val P

roce

sses

1. Advanced knowledge, skills, values X X X X X X X X X X

2. Creative, analytical, critical thinking X X X X X X X X X X X X

3. Individual and collaborative scholarship X X X X X X X X X X X

4. Global perspectives X X X X X X X X X X X

5. Methods and technologies X X X X X X X X X

6. Communication skills; source analysis X X X X X X X X X X X X X X

*Direct methods. Table excerpted from the CSU Stanislaus Graduate Assessment Plan (2009).

The three levels of student learning assessment at CSU Stanislaus are institution-level, program-level, and course-

level. Institution-level graduate learning goals provide a structure for institution-wide learning outcomes

assessment. These aims, stated in the graduate student learning goals, collectively specify that all graduate

students, regardless of what program they pursue, will be taught and assessed in the knowledge, skills and

dispositions defined in these aims.

Table 2 illustrates the way in which graduate learning goals could be used as the “anchor” in an alignment process,

whereby program learning outcomes are listed and aligned with graduate learning goals. Similarly, courses that

fulfill these learning outcomes are also listed and therefore in alignment with the graduate learning goals. An

assessment conducted in a course contributes to determining the level of student competency in the program and

towards achieving the graduate learning goals.




Table 2 Alignment of Graduate Student Learning Goals, Program Learning Outcomes, and Courses

Graduate

Learning

Goals

Advanced

knowledge,

skills, and

values

Creative,

analytical, and

critical thinkers.

Researcher/

Scholar

Global

Perspectives

Knowledge of

new and

various

methods and

technologies

Advanced oral

and written

communication

/information

literacy

Program

Learning

Outcomes

e.g., PLO 1 PLO 3 PLO 2,4 PLO 5 PLO 1,5 PLO 6

Courses e.g., Course

X Course Y Course Z Course A Course B Course C

Graduate Council: Reviewing and Reporting on Graduate Assessment Results

Graduate Council reviews all graduate-level APRs. Information is collected from the programs on student

achievement on the six learning goals.

Adapted from Draft Institutional Plan for the Assessment of Graduate Programs (2012).

Appendix 4 California State University, Stanislaus Academic Program Review Timeline


TARGET DATE YEAR 6 ACTIVITY

By February 1 AVP (Academic Affairs) notifies college deans and department chairs/program administrators the programs to

be reviewed two years prior to the completion date of the self study, recommendations, and implementation

plan.

By March 1 Department chair/program administrator request from the college dean that the program be subject to an

external evaluation. An external consultant to be invited to assist in the self-study phase of the academic

program review process.

By March 15 AVP (Academic Affairs) conducts a program review workshop(s) with department chairs/program

administrators and program faculty to discuss the academic program review process and discuss data provided

by institutional research, as required for the academic program review.

March 16 – May 29 Department chair and dean identify process and timeline for milestones for completion and identify/arrange for

external reviewers (as appropriate).

March 16 – May 29 Program faculty and department chair begin draft review of data and begin draft of self study.

March 16 (Year 6) –

February 1 (Year 7)

Program faculty and department chair conduct the self study and complete the self-study document, including

recommendations and a preliminary implementation plan.

TARGET DATE YEAR 7 ACTIVITY

By February 1 Department chair/program administrator submits the self study and supporting materials to the college dean.

February 1 – 27 College dean submits self study to external reviewers (as appropriate).

February 15 –

April 30

College governance committee(s) reviews the self study, requests additional materials as needed, summarizes

findings, and forwards the self study to the department chair/program administrator.

February 15 –

April 30

General Education Subcommittee reviews the General Education portion of the self study, summarizes findings,

and forwards the recommendations for recertification of the GE curriculum (lower- and upper-division) to the

department chair/program administrator.

By April 30 College dean forwards the self study to the Office of Academic Programs.

By April 30 Office of Academic Programs forwards the self study to the UEPC (if requested) and/or to the Graduate Council

(for master’s and post-baccalaureate programs).

April 30 – May 29 UEPC and/or Graduate Council (as appropriate) reviews the self study, summarizes the findings, and forwards

the document and findings to the department chair/program administrator and college dean.

May 29 – June 30 College dean finalizes self study to include recommendations from external reviewer(s) (if applicable); responses

from the department (if any); recommendations from the college governance committee(s), UEPC, and/or

Graduate Council; and dean’s recommendation for program continuance, continuance with conditions, or

program discontinuance.

By June 30 College dean submits to the AVP the self study; recommendations from external reviewer(s) (if applicable);

responses from the department (if any); recommendations from the college governance committee(s), UEPC,

and/or Graduate Council; and dean’s recommendation for program continuance, continuance with conditions,

or program discontinuance.

Appendix 4 California State University, Stanislaus Academic Program Review Timeline


TARGET DATE YEAR 7 ACTIVITY continued

September –

October

Office of Academic Programs schedules a meeting to include the program representative(s), the department

chair/program administrator, the college dean, the AVP, and the provost to discuss the results of the academic

program review and the preliminary implementation plan.

October –

November

Department chair/program administrator submits to the college dean a final implementation plan that

identifies resource needs consistent with the recommendations of reviewing committees and consistent with

the college mission and strategic plan. Within three weeks, the college dean submits the final implementation

plan to the AVP.

By December 1 Provost issues a letter indicating final determination of program continuance and additionally may require

progress reports and a timeline related to specific elements of the final implementation plan.

By December 15 Office of Academic Programs archives the academic program review documents and posts on the web (program

faculty’s final implementation plan and provost’s recommendation for program continuance/discontinuance).

By January 15 AVP provides a summary of academic program reviews to the Board of Trustees.

ONGOING College dean incorporates the results of the academic program review into the college’s strategic and budget

planning processes and forwards to the provost as part of the regular planning and budgetary processes within

academic affairs and within the university’s strategic planning processes.

Department chair/program administrator submits annual reports (due September) based on program review

implementation plan to college dean.

ACCREDITED

PROGRAMS

Accredited programs will follow the applicable timeline steps as they apply to the accrediting agency’s due date

and timeline for the program’s self study.

After completion of the accreditation review, and upon receipt of the accrediting agency’s determination letter,

the Department chair/program administrator requests of the college dean a substitution for the academic

program review document.

College dean determines whether the accreditation review process fulfills all or a portion of the academic

program review in accordance with any CSU or CSU Stanislaus mandated requirements and communicates

decision to the department chair/program administrator. College dean submits completed substitution for the

academic program review documents to the Office of Accreditation.

Office of Accreditation schedules a meeting to include the program representative(s), the department

chair/program administrator, the college dean, the AVP, and the provost to discuss the results of the

accreditation self study and the recommendations. If periodic reporting is not required by the accrediting

agency, an implementation plan will be developed for the university’s annual reporting process.


Curriculum Map Template


Program Learning Outcomes

Core Courses

E.g.

, Co

urs

e A

Co

urs

e B

Co

urs

e C

Co

urs

e D

Co

urs

e E

Co

urs

e F

Co

nti

nu

e…

E.g. Written Communication

Skills I E

Oral Communication Skills

I

Information Literacy

E

Critical Thinking

I E R

Quantitative Literacy

E

KEY:

Outcomes assessment data not collected for program-level

analysis

Outcome assessment data collected for program-level analysis

NOTE: In the curriculum map, the level at which the outcome is addressed is identified. In this case, the following

scale is used: Introduced (I), Emphasized (E), and Reinforced (R). Other options include:

Introduced (I), Developed (D), and Mastered (M)

Introduced (I), Practiced (P), Demonstrated (D)

Introduced (I), Reinforced (R), and Mastered (M)

Basic (B), Intermediate (I), Advanced (A)


External Consultant Review


External Consultant’s Letter of Invitation DATE Dr. XXX XXX Address Dear Dr. XXX: I am writing to invite you to serve as external consultant for California State University, Stanislaus’ XXX program (BA, BS). As part of CSU Stanislaus’ assessment effort, every academic program undergoes an Academic Program Review every seven years. The review is intended to help us identify a program’s strengths and areas in which it could improve; determine the program’s education effectiveness by assessing student learning; and provide data for informed planning. Dr. XXX, Professor of XXX and Department Chair has suggested you as someone who could help us with that task. The role of the external consultant is to provide an objective assessment of the quality and effectiveness of the academic program, resources, and operations based on the program’s self-study and questions asked by program faculty and administration. We would ask you to read the XXX program self-study; visit CSU Stanislaus sometime this fall to meet with members of the department and administration as well as students; and write a report based on the self study and interviews, responding to specific questions that faculty and administration might ask as well as noting what a program is doing well and making overall recommendations for quality enhancement. I have attached three documents that should help address questions that you might have about the review: 1) the template for the external consultant’s report; 2) a typical schedule; and 3) a document describing the purpose of the review. The XXX Department has prepared some specific questions that I will send should you accept this invitation. In addition, we would send the departmental study and additional questions that I would ask you to address. CSU Stanislaus will pay your travel expenses and an honorarium of $250 per day. I will follow up this invitation next week with a phone call to see if you are able to do this and to answer any questions that you might have; please feel free to call or write in the meantime. If you are able to do this, I will ask Mr. XXX, my administrative assistant, to call you to schedule your visit and help you make travel arrangements. We would be very pleased if you are able to do this. Thank you for considering the invitation. I look forward to hearing from you. Sincerely, XXX Dean, College of XX




External Consultant’s Sample Schedule

Program Date

Time Interview / Meeting Location

Date Evening Dinner with Faculty / Dean / Consultant

Restaurant

Date

8:30 – 9:30 am

Program Department Chair Tour of Campus / Facilities

9:30 – 10:00 am Meet with Dean

10:00 – 10:30 am Meet with AVPAA

10:30 – 11:00 am

Meet with Faculty

11:00 am – Noon Classroom Observation

Noon – 1:00 pm Lunch with Students

1:00 – 3:00 pm Meet with Faculty

3:00 – 4:00 pm Meet with Service Learning / Librarian / Institutional Research

4:00 – 4:30 pm Meet with Alumni

4:30 – 5:00 pm Meet with Faculty to deliver Preliminary Impressions and Report

5:00 pm Departure

**include time for external consultant to meet with Human Resources and sign paperwork (if employed by the CSU)




External Consultant’s Report Template

Program External Consultant’s University Date This template is provided to assist External Consultants in the completion of their report. External Consultants are asked to write a report that:

Includes an executive summary of general comments, singles out features of the program that merit commendations, and makes recommendations for improvement.

Varies in length between three and five pages.

Is completed within two weeks of the visit. 1. Executive Summary Provide a brief executive summary of major findings for this program. Include:

General observations and comments on the program and curriculum, quality of student learning and the achievement of student learning outcomes, the implementation plan, faculty, students, facilities, and resources

Reponses to questions posed by faculty 2. Commendations Provide comments about what the program is doing well. Note suggested topic areas below. 3. Recommendations Provide comments to guide future direction for faculty to use to improve student learning. Provide evaluative feedback that would improve any aspect of the program and recommendations that require no new resources as well as those that do. The report may note recommendations that have been shown to be effective elsewhere. Note suggested topic areas below. Educational Effectiveness Topic Areas for Commendations and Recommendations sections:

Provide feedback / suggestions on any learning outcome.

Analyze / evaluate direct and indirect evidence of student learning

Offer suggestions to improve the assessment process

Evaluate assessment plan

Evaluate assessment impact




EXTERNAL CONSULTANT FUNDING REQUEST FORM

Programs may apply for up to $1,000 to assist with an external consultant’s fees and travel expenses. As noted in the Academic Program Review Procedures, the college dean and chair of the department coordinate the external consultant’s travel arrangements and visit schedule, in accordance with University travel policy. A consultant contract is issued to the external consultant (normally $250 per day), plus transportation and one-night lodging, as required. The payment and refunds are processed upon receipt of the written report from the external consultant and documented accommodation and travel costs, as previously approved. Departments will be reimbursed for approved external consultant expenses upon receipt of a copy of the External Consultant’s report to the Office of Assessment. For additional information on the required qualifications and selection of an external consultant, see Academic Program Review (APR) Procedures Section VI: External Consultant Review.

Completed request forms can be submitted to the Office of Assessment - MSR 376.

Today’s Date _____________________________

College __________________________________ Department/Program ____________________________

Primary Contact ____________________________________________________________________________

Phone ________________________________ Email _________________________________________

Budget Summary

Item Description

e.g., travel

Total Budget Requested




Signatures

Department Chair ________________________________ Date_____________________________

Dean_____________________________________________ Date_____________________________

AVP (Academic Affairs)________________________________ Date_____________________________

Timeline

Action Month / Date Person Responsible


Seven Year Implementation Plan Template


College: Program: Next APR Year:

Mission Statement:

[Enter your program Mission Statement here]

Program Learning Outcomes: PLO 1: PLO 2: Etc.

Program Maintenance Outcomes: PMO 1: PMO 2: Etc.

Where are these outcomes published? [e.g., University Catalog, Department website]

Please attach the most current program curriculum map.

SEVEN YEAR IMPLEMENTATION PLAN TEMPLATE

INITIATIVE/TASK APR

CYCLE

YEAR ASSESSMENT METHODS/ACTIVITIES

ASSESSMENT PROCESS: COLLECTION/ANALYSIS/ DISCUSSION

RESOURCES NEEDED (IF APPLICABLE)

Program Learning Outcome (PLO)

PLO 1:

PLO 2:

PLO 3:

PLO 4:

Program Maintenance Outcome (PLO)

PMO 1:

PMO 2:

PMO 3:

PMO 4:

Appendix 8 California State University Stanislaus

Provost's Academic Program Review Meetings


This template provides an overview of the provost's Academic Program Review meeting. Participants include the department chair, program coordinator and/or faculty (as determined by department chair or dean), dean, and the associate vice president (AVP).

INTRODUCTIONS (DEAN) 1. Dean's introductions of meeting participants.

OVERVIEW (DEAN AND DEPARTMENT CHAIR) 2. Dean's overall evaluation of program and key issues identified by the college's committee and dean. 3. Department chair's overview of major findings/issues.

PROGRAM QUALITY (PROVOST) 4. Provost's questions/discussions regarding the self-study's findings. 5. Overall conclusions about program quality and assessment of student learning. 6. Implementation Plan:

a. Program Learning Outcomes b. Program Maintenance Outcomes

NEXT STEPS (AVP) 7. Explanation of next steps

a. Department chair completes final Implementation Plan and submits to the dean; dean submits electronic document to AVP (within 2 weeks following meeting).

b. Upon receipt and review of the final Implementation Plan, Provost issues letter indicating final determination for program continuance.

EVALUATION OF THE ACADEMIC PROGRAM REVIEW PROCESS (AVP) 8. In what ways did the faculty find this academic program review process helpful for program improvement? 9. What are your recommendations for improving the academic program review process?

a. Review Criteria, especially student learning b. Internal/External review components c. Institutional research/assessment data d. Timeline e. Department/college review process and report f. Other

CLOSING COMMENTS (PROVOST)


Annual Report Template


College: Program: Next APR year: Program Learning Outcomes: PLO 1: PLO 2: Etc.

Program Maintenance Outcomes: PMO 1: PMO 2: Etc.

Assessed Outcomes Assessment Methods Data Reviewed and Findings Actions

(List PLO# and/or PMO# for outcomes assessed during the previous AY)

(Describe the assessment methods used this year and indicate the targeted PLO by #)

(Provide a description of the data reviewed and a summary of the findings. Describe the process for evaluating/analyzing the findings)

(Describe implemented or planned actions based on findings)

Approvals: Department Chair/ Program Director ___________________________Date__________ College Dean ___________________________ Date _________

DATE DUE: September 20, 2012


Degree Audit Information


Department

Program Title and Degree

Line

Proposed Program

(# of units)

Description

1 51 University general education requirements

2 Prerequisites to the major

3 Upper-division (major requirements)

4 WP course (if not required in the major)

5 Other (if applicable)

6 TOTAL minimum units required (add lines 1 through 5)

7 University elective units (subtract line 6 from line 8)

8 120 TOTAL UNIT DEGREE REQUIREMENTS

9 WP course required in the major

Course prefix and number:

10

Lower-division prerequisite course(s) that may be applied toward GE

Course prefix, number, units, area:






11 TOTAL double-counted courses (add lines 9 and 10)

12 TOTAL units taken (subtract line 11 from line 8)

Prepared by Date

Approved by Date

Documents

Academic Program Review SELF-STUDY APPENDICES of... · Appendix 2 California State University, Stanislaus Substitution of Specialized Accreditation Self Study for the Academic Program