69
AY 2012 CTE AY 2012 CTE Instructional Program Instructional Program Review Process Review Process Data Description Data Description Process – Timeline Process – Timeline Rev. 10-15-12 Rev. 10-15-12

AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Embed Size (px)

Citation preview

Page 1: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

AY 2012 CTE Instructional AY 2012 CTE Instructional Program Review ProcessProgram Review Process

Data DescriptionData DescriptionProcess – TimelineProcess – Timeline

Rev. 10-15-12Rev. 10-15-12

Page 2: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

PurposePurpose

The purpose of this presentation is describe the process we follow for our local Comprehensive Program The purpose of this presentation is describe the process we follow for our local Comprehensive Program Review, the system required Annual Program Reviews, and to describe how the data is derived.Review, the system required Annual Program Reviews, and to describe how the data is derived.

We have been asked to produce an annual program review for each and every one of our instructional We have been asked to produce an annual program review for each and every one of our instructional programs and units. They are required of each community college in the system and will be taken to the U of programs and units. They are required of each community college in the system and will be taken to the U of H Board of Regents for their purview.H Board of Regents for their purview.

If you are normally scheduled to do a comprehensive review or are “Jumping”, you will need to complete a If you are normally scheduled to do a comprehensive review or are “Jumping”, you will need to complete a comprehensive review this year. Additionally, every instructional and non-instructional program will do an comprehensive review this year. Additionally, every instructional and non-instructional program will do an Annual Program Review this year.Annual Program Review this year.

Not sure if you’re scheduled for a comprehensive review this year? Not sure if you’re scheduled for a comprehensive review this year?

Find out by clicking here for theFind out by clicking here for the

Comprehensive Program-Unit Review Cycle and ScheduleComprehensive Program-Unit Review Cycle and Schedule

22

Page 3: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Reason for Program ReviewReason for Program Review

Program Review Program Review isis Assessment Assessment

The review of a program should be an on-going, year-round, reflective process. The review of a program should be an on-going, year-round, reflective process. 

Program review processes assure educational quality and help programs to evaluate and improve their Program review processes assure educational quality and help programs to evaluate and improve their services.services.

Program review is an opportunity for self-study, self-renewal, and an opportunity to identify the need for Program review is an opportunity for self-study, self-renewal, and an opportunity to identify the need for improvement. improvement.

A robust program review process is one mechanism available to the college to improve student success.A robust program review process is one mechanism available to the college to improve student success.

33

Page 4: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

What are we doing to improve our program review process?What are we doing to improve our program review process?

Upon conclusion of every program/unit review cycle, the IR Office takes extra care to ensure that we are improving our program/unit review Upon conclusion of every program/unit review cycle, the IR Office takes extra care to ensure that we are improving our program/unit review process on campus. This is accomplished by sending out questionnaires specific to the groups, and by meeting with the various groups process on campus. This is accomplished by sending out questionnaires specific to the groups, and by meeting with the various groups across campus and collecting their feedback. Your suggestions for improving this process are then published to the Program Review across campus and collecting their feedback. Your suggestions for improving this process are then published to the Program Review website and have been linked here for your convenience.website and have been linked here for your convenience.

AY 2011 Program/Unit Process Improvement SummaryAY 2011 Program/Unit Process Improvement Summary

Based upon the feedback we received from everyone last year from our program-unit review process improvement focus groups, the Based upon the feedback we received from everyone last year from our program-unit review process improvement focus groups, the following changes have been made and have been incorporated into the planning of this year’s review:following changes have been made and have been incorporated into the planning of this year’s review:

In order to assure the best possible attendance for our annual training, the VC for Academic Affairs will notify the Chancellor, VC’s, and In order to assure the best possible attendance for our annual training, the VC for Academic Affairs will notify the Chancellor, VC’s, and Directors, when training is ready, and the training will be scheduled through the VCAA’s office and Secretaries. It will be the responsibility of Directors, when training is ready, and the training will be scheduled through the VCAA’s office and Secretaries. It will be the responsibility of the VC’s and Directors to disseminate the day, time, and location of training to Writers and Initiators.the VC’s and Directors to disseminate the day, time, and location of training to Writers and Initiators.

In order to improve navigation on the program review website, our web developer has reorganized the current site and will be responsible for In order to improve navigation on the program review website, our web developer has reorganized the current site and will be responsible for publishing documentation going forward.publishing documentation going forward.

To provide the best possible support for training on campus we will continue to have 3 separate training sessions this year, one for Units, To provide the best possible support for training on campus we will continue to have 3 separate training sessions this year, one for Units, one for Liberal Arts, and one for CTE programs.one for Liberal Arts, and one for CTE programs.

44

Page 5: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

What else are we doing to improve our program review process?What else are we doing to improve our program review process?

There were some performance issues with the online tool last year reported by Writers. Joni has taken your concerns to the appropriate There were some performance issues with the online tool last year reported by Writers. Joni has taken your concerns to the appropriate party at the system office and has asked them to try to work some of the issues out.party at the system office and has asked them to try to work some of the issues out.

You said that the training you received last year regarding changes to the Instructional Program Review Comprehensive Template was You said that the training you received last year regarding changes to the Instructional Program Review Comprehensive Template was helpful. We will continue to provide a brief overview of the templates upon conclusion of our normally scheduled training.helpful. We will continue to provide a brief overview of the templates upon conclusion of our normally scheduled training.

All annual and comprehensive templates have been updated since last year based on your feedback, and on an evaluation of the All annual and comprehensive templates have been updated since last year based on your feedback, and on an evaluation of the comprehensive review templates by CERC.comprehensive review templates by CERC.

An additional step to our Comprehensive Program/Unit Review Process document was added to ensure timely delivery of the budgetary An additional step to our Comprehensive Program/Unit Review Process document was added to ensure timely delivery of the budgetary data needed to complete an instructional program review.data needed to complete an instructional program review.

An additional step to our Comprehensive Program/Unit Review Process document was added to ensure that VC’s and Directors took more An additional step to our Comprehensive Program/Unit Review Process document was added to ensure that VC’s and Directors took more ownership for our local program/unit review process by providing training specific to their groups (i.e. the Vice Chancellor for Academic ownership for our local program/unit review process by providing training specific to their groups (i.e. the Vice Chancellor for Academic Affairs would provide training to the Academic Support Group)Affairs would provide training to the Academic Support Group)

An additional step to our Comprehensive Program/Unit Review Process document was added to ensure that all of the suggested An additional step to our Comprehensive Program/Unit Review Process document was added to ensure that all of the suggested improvements we commit to every year through our Program/Unit Process Improvement efforts are completed by the responsible party improvements we commit to every year through our Program/Unit Process Improvement efforts are completed by the responsible party prior to leaving for Summer.prior to leaving for Summer.

This is the This is the improvedimproved process we are using this year: process we are using this year:

Comprehensive Program-Unit Review Process Comprehensive Program-Unit Review Process

55

Page 6: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

What is different this year?What is different this year?

Program Student Learning Outcomes are now required for all CTE Programs in the Program Student Learning Outcomes are now required for all CTE Programs in the community college system.community college system.

Under the online Web Submission “External” tab there is a new requirement for the Under the online Web Submission “External” tab there is a new requirement for the entry of the number sitting and number passing External Licensing Exams.entry of the number sitting and number passing External Licensing Exams.

Under the online Web Submission “Cost per SSH” tab there is a new requirement for Under the online Web Submission “Cost per SSH” tab there is a new requirement for the entry of Tuition and fees, which are now part of the Overall Program Budget the entry of Tuition and fees, which are now part of the Overall Program Budget Allocation for all instructional programs. Joni will be uploading all of the budget Allocation for all instructional programs. Joni will be uploading all of the budget information you need this year.information you need this year.

Under the online Web Submission “Description” tab there is a new requirement for Under the online Web Submission “Description” tab there is a new requirement for entry of the web address to the location of your last comprehensive review. Also entry of the web address to the location of your last comprehensive review. Also required is the date completed. Please link your last comprehensive review here—required is the date completed. Please link your last comprehensive review here—not this year’s comprehensive.not this year’s comprehensive.

66

Page 7: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

What else is different this year?What else is different this year?

Academic Subject Certificates were included last year as part of your count for Academic Subject Certificates were included last year as part of your count for certificates awarded. This year it is grouped under “other certificates awarded.”certificates awarded. This year it is grouped under “other certificates awarded.”

There is a different certificate included in your data this year as part of the count for There is a different certificate included in your data this year as part of the count for certificates awarded. It is the Advanced Professional Certificate.certificates awarded. It is the Advanced Professional Certificate.

Annual Reports of Program Data are completely on-line this year for Academic Annual Reports of Program Data are completely on-line this year for Academic Support.Support.

The number of new and replacement positions for your programs this year will be The number of new and replacement positions for your programs this year will be determined by the CIP code assigned by IRAO.determined by the CIP code assigned by IRAO.

Data elements with asterisks next to them are now used for health calls.Data elements with asterisks next to them are now used for health calls.

In the online Web Submission section there is a new “P-SLO” tab, which requires the In the online Web Submission section there is a new “P-SLO” tab, which requires the following new information: Evidence of Industry Validation (for CTE programs), following new information: Evidence of Industry Validation (for CTE programs), Expected Level of Achievement, Courses Assessed, Assessment Expected Level of Achievement, Courses Assessed, Assessment Strategy/Instrument, Results of Program Assessment, Other Comments, and Next Strategy/Instrument, Results of Program Assessment, Other Comments, and Next Steps.Steps. (see following slide for definitions)(see following slide for definitions)

77

Page 8: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

What to enter under the P-SLO tabWhat to enter under the P-SLO tab

Evidence of Industry ValidationEvidence of Industry Validation

Provide documentation that the program has submitted evidence and achieved certification or accreditation from an Provide documentation that the program has submitted evidence and achieved certification or accreditation from an organization granting certification in an industry or profession.  If the program/degree/certificate does not have a organization granting certification in an industry or profession.  If the program/degree/certificate does not have a certifying body, the recommendations for, approval of, and/or participation in, assessment by the program’s advisory certifying body, the recommendations for, approval of, and/or participation in, assessment by the program’s advisory committee/board can be submitted.committee/board can be submitted.

  

Expected Level of AchievementExpected Level of Achievement

Describe the different levels of achievement for each characteristic of the learning outcome(s) that were assessed.  Describe the different levels of achievement for each characteristic of the learning outcome(s) that were assessed.  What represented “excellent,” “good,” “fair,” or “poor” performance using a defined rubric and what percentages What represented “excellent,” “good,” “fair,” or “poor” performance using a defined rubric and what percentages were set as goals for student success (for example: “85% of students will achieve good or excellent in the assessed were set as goals for student success (for example: “85% of students will achieve good or excellent in the assessed activity.”)activity.”)

Courses AssessedCourses Assessed

List the courses assessed during the reporting period.List the courses assessed during the reporting period.

  

Assessment Strategy/InstrumentAssessment Strategy/Instrument

Describe what, why, where, when, and from whom assessment artifacts were collected.Describe what, why, where, when, and from whom assessment artifacts were collected.

Results of Program AssessmentResults of Program Assessment

The % of students who met the outcome(s) and at what level they met the outcome(s).The % of students who met the outcome(s) and at what level they met the outcome(s).

Other CommentsOther Comments

Include any information that will clarify the assessment process report.Include any information that will clarify the assessment process report.

  

Next StepsNext Steps

Describe what the program will do to improve the results.  "Next Steps" can include revision to syllabi, curriculum, Describe what the program will do to improve the results.  "Next Steps" can include revision to syllabi, curriculum, teaching methods, student support, and other options.teaching methods, student support, and other options.

88

Page 9: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Jumper DefinedJumper Defined

A Jumper is a locally defined term that is used to describe an instructional A Jumper is a locally defined term that is used to describe an instructional program or non-instructional program (unit) that has decided to jump out of program or non-instructional program (unit) that has decided to jump out of their normally scheduled slot for their comprehensive reviews and into this their normally scheduled slot for their comprehensive reviews and into this years cycle.years cycle.

Jumping into this years comprehensive cycle means that you will have an Jumping into this years comprehensive cycle means that you will have an opportunity to be considered for any budgetary decisions that will be made opportunity to be considered for any budgetary decisions that will be made in this years budget process.in this years budget process.

Jumpers will still have to do their comprehensive review on their next Jumpers will still have to do their comprehensive review on their next scheduled review. Jumping does not affect the existing schedule—you are scheduled review. Jumping does not affect the existing schedule—you are voluntarily doing an extra review to be considered in this budget cycle.voluntarily doing an extra review to be considered in this budget cycle.

99

Page 10: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

I belong to an Instructional Program… which template do I use?I belong to an Instructional Program… which template do I use?

COMPREHENSIVE REVIEWSCOMPREHENSIVE REVIEWS

Comprehensive Instruction Program Review TemplateComprehensive Instruction Program Review Template

(Use this template ONLY if you are scheduled for a comprehensive program review this year or are jumping)(Use this template ONLY if you are scheduled for a comprehensive program review this year or are jumping)

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ANNUAL REVIEWSANNUAL REVIEWS

Your programs data table is available on-line using the link below. You should have everything you need Your programs data table is available on-line using the link below. You should have everything you need to begin writing your review within the web submission tool. Plan to save your work often—especially to begin writing your review within the web submission tool. Plan to save your work often—especially when switching between screens, and plan to do most of your formatting within the tool if you are copying when switching between screens, and plan to do most of your formatting within the tool if you are copying and pasting in from Word.and pasting in from Word.

UHCC Annual Report of Program Data Web Submission ToolUHCC Annual Report of Program Data Web Submission Tool

((ALLALL Instructional programs will need to complete this—even if you’re completing a comprehensive review Instructional programs will need to complete this—even if you’re completing a comprehensive review this year)this year)

1010

Page 11: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Terminology / TimingTerminology / Timing

The Census freeze event is the fifth Friday after the first day of instruction.The Census freeze event is the fifth Friday after the first day of instruction.

The End of semester freeze event is 10 weeks after the last day of The End of semester freeze event is 10 weeks after the last day of instruction.instruction.

FISCAL_YR_IRO: Fiscal year, where the value indicates the ending of the FISCAL_YR_IRO: Fiscal year, where the value indicates the ending of the fiscal year. For example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year. For example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004-2005 (July 1, 2004 to June 30, 2005) which includes fiscal year 2004-2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2004, and Spring 2005 semesters. Summer 2004, Fall 2004, and Spring 2005 semesters.

1111

Page 12: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Instructional Program Review Data ElementsInstructional Program Review Data Elements

Student information data this year comes exclusively from the Operational Data Student information data this year comes exclusively from the Operational Data Store (ODS). Organizationally this means that all community colleges are Store (ODS). Organizationally this means that all community colleges are getting the data from the same place and at the same snapshot in time (this is a getting the data from the same place and at the same snapshot in time (this is a good thing).good thing).

The following slides will explain in detail what data has been provided to you for The following slides will explain in detail what data has been provided to you for your comprehensive-annual instructional program review write ups and how it your comprehensive-annual instructional program review write ups and how it has been calculated.has been calculated.

1212

Page 13: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#1#1 New and Replacement Positions (State)New and Replacement Positions (State)

Economic Modeling Specialists Inc. (EMSI) annual new and replacement jobs at Economic Modeling Specialists Inc. (EMSI) annual new and replacement jobs at state level. Compiles data based on Standard Occupational Classification (SOC) state level. Compiles data based on Standard Occupational Classification (SOC) codes aligned to the program’s Classification of Instructional Programs (CIP) codes.codes aligned to the program’s Classification of Instructional Programs (CIP) codes.

Data based on annual new/replacement position projections as of Spring 2012Data based on annual new/replacement position projections as of Spring 2012

State position numbers are not pro‐rated. State position numbers are not pro‐rated.

Note: In the past programs could select SOC codes and some programs had more Note: In the past programs could select SOC codes and some programs had more than one.  This year they are using the official CIP code assigned by IRAO. Click on than one.  This year they are using the official CIP code assigned by IRAO. Click on the link below for a list of the CIP codes that were used to pull the labor data.  The the link below for a list of the CIP codes that were used to pull the labor data.  The SOC codes that are associated with this CIP code is actually where the data is SOC codes that are associated with this CIP code is actually where the data is coming from.coming from.

AY 2012 HawCC CIP Code Listing AY 2012 HawCC CIP Code Listing

1313

Page 14: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#2#2 New and Replacement Positions (County prorated) New and Replacement Positions (County prorated)

Economic Modeling Specialists Inc. (EMSI) annual new and replacement jobs Economic Modeling Specialists Inc. (EMSI) annual new and replacement jobs at county level. Compiles data based on Standard Occupational Classification at county level. Compiles data based on Standard Occupational Classification (SOC) codes aligned to the program’s Classification of Instructional Program (SOC) codes aligned to the program’s Classification of Instructional Program (CIP) codes.(CIP) codes.

Note: It is possible for the number of new and replacement positions in the Note: It is possible for the number of new and replacement positions in the county to be higher than the state if the projection in other counties is for a loss county to be higher than the state if the projection in other counties is for a loss of new and replacement positions.of new and replacement positions.

County data pro‐rated to reflect number of programs aligned to the SOC code County data pro‐rated to reflect number of programs aligned to the SOC code and weighted by number of majors in each program/institution for programs and weighted by number of majors in each program/institution for programs that share SOC codes.that share SOC codes.

Data based on annual new/replacement position projections as of Spring Data based on annual new/replacement position projections as of Spring 2012.2012.

1414

Page 15: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#3#3 Number of MajorsNumber of Majors

Count of program majors who are home‐institution at your college. Count Count of program majors who are home‐institution at your college. Count excludes students that have completely withdrawn from the semester at excludes students that have completely withdrawn from the semester at CENSUS.CENSUS.

This is an annual number. Programs receive a count of .5 for each term (fall This is an annual number. Programs receive a count of .5 for each term (fall and spring) within the academic year that the student is a major. A and spring) within the academic year that the student is a major. A maximum count of 1.0 (one) for each student.maximum count of 1.0 (one) for each student.

1515

Page 16: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#4#4 SSH Program majors in Program ClassesSSH Program majors in Program Classes

The sum of Fall and Spring SSH taken by program majors in courses linked The sum of Fall and Spring SSH taken by program majors in courses linked to the program. Captured at Census and excludes students who have to the program. Captured at Census and excludes students who have already withdrawn (W) at this point.already withdrawn (W) at this point.

Note: for programs where year‐round attendance is mandatory, Summer Note: for programs where year‐round attendance is mandatory, Summer SSH are included.SSH are included.

Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Excludes Directed Studies (99 series). Differs from MAPS as UHCC data includes Cooperative Education (93 series) as there is a resource cost to includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

Not sure what your program classes are? Click here to find out:Not sure what your program classes are? Click here to find out:

Courses Taught Aligned to Instructional ProgramsCourses Taught Aligned to Instructional Programs

1616

Page 17: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#5#5 SSH Non-Majors in Program ClassesSSH Non-Majors in Program Classes

The sum of Fall and Spring SSH taken by non‐program majors (not counted The sum of Fall and Spring SSH taken by non‐program majors (not counted in #4) in courses linked to the program. Captured at Census and excludes in #4) in courses linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point.students who have already withdrawn (W) at this point.

Note: for programs where year‐round attendance is mandatory, Summer Note: for programs where year‐round attendance is mandatory, Summer SSH are includedSSH are included

Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to Includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

1717

Page 18: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#6#6 SSH in All Program ClassesSSH in All Program Classes

The sum of Fall and Spring SSH taken by all students in classes linked to The sum of Fall and Spring SSH taken by all students in classes linked to the program. Captured at Census and excludes students who have already the program. Captured at Census and excludes students who have already withdrawn (W) at this point.withdrawn (W) at this point.

Note: for programs where year‐round attendance is mandatory, Summer Note: for programs where year‐round attendance is mandatory, Summer SSH are included.SSH are included.

Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to Includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

1818

Page 19: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#7#7 FTE Enrollment in Program ClassesFTE Enrollment in Program Classes

Sum of Student Semester Hours (SSH) taken by all students in classes Sum of Student Semester Hours (SSH) taken by all students in classes linked to the program (#6) divided by 30. Undergraduate, lower division Full linked to the program (#6) divided by 30. Undergraduate, lower division Full Time Equivalent (FTE) is calculated as 15 credits per term. Time Equivalent (FTE) is calculated as 15 credits per term.

Captured at Census and excludes students who have already withdrawn Captured at Census and excludes students who have already withdrawn (W) at this point.(W) at this point.

Note: for programs where year‐round attendance is mandatory, summer Note: for programs where year‐round attendance is mandatory, summer SSH are included.SSH are included.

1919

Page 20: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#8#8 Total Number of Classes TaughtTotal Number of Classes Taught

Total number of classes taught in Fall and Spring that are linked to the Total number of classes taught in Fall and Spring that are linked to the program. Includes Summer classes if year‐round attendance is mandatory.program. Includes Summer classes if year‐round attendance is mandatory.

Concurrent and Cross listed classes are only counted once for the primary Concurrent and Cross listed classes are only counted once for the primary class. class.

Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Excludes Directed Studies (99 series). Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to Includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

2020

Page 21: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

CTE Program Scoring Rubric DefinitionsCTE Program Scoring Rubric Definitions

Your program health is determined by 3 separate types of measures: Your program health is determined by 3 separate types of measures: Demand, Efficiency, and Effectiveness. This slide explains why these Demand, Efficiency, and Effectiveness. This slide explains why these measures were chosen to determine program health.measures were chosen to determine program health.

Demand: Demand: A seeking or state of being sought after.A seeking or state of being sought after.

i.e. your programs ability to attract new students every year based on your i.e. your programs ability to attract new students every year based on your offering.offering.

Efficiency: Efficiency: Acting or producing effectively with a minimum of waste, Acting or producing effectively with a minimum of waste, expense, or unnecessary effort.expense, or unnecessary effort.

i.e. your programs ability to use its resources in the best possible way.i.e. your programs ability to use its resources in the best possible way.

Effectiveness: Effectiveness: Stresses the actual production of, or the power to produce an Stresses the actual production of, or the power to produce an affect.affect.

i.e. your programs ability to produce the desired result.i.e. your programs ability to produce the desired result.

2121

Page 22: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on demanddemand

This year the system office will calculate and report health calls for all This year the system office will calculate and report health calls for all instructional programs using academic year 2012 data. The following instructional programs using academic year 2012 data. The following instructions illustrate how those calls are made.instructions illustrate how those calls are made.

Program Demand is determined by taking the number of majors (#3) and Program Demand is determined by taking the number of majors (#3) and dividing them by the number of New and Replacement Positions by County dividing them by the number of New and Replacement Positions by County (#2).(#2).

The following benchmarks are used to determine demand health:The following benchmarks are used to determine demand health:

Healthy:Healthy: 1.5 - 4.01.5 - 4.0Cautionary:Cautionary: .5 – 1.49; 4.1 – 5.0.5 – 1.49; 4.1 – 5.0Unhealthy:Unhealthy: <.5; >5.0<.5; >5.0

Finally, an Overall Category Health Score is assigned where:Finally, an Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1= Cautionary1= Cautionary0= Unhealthy0= Unhealthy

2222

Page 23: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#9#9 Average Class SizeAverage Class Size

Total number of students actively registered in Fall and Spring program Total number of students actively registered in Fall and Spring program classes divided by classes taught (#8). Does not include students who have classes divided by classes taught (#8). Does not include students who have already withdrawn from the class by Census.already withdrawn from the class by Census.

Excludes Directed Studies (99 series). Differs from MAPS as UHCC data. Excludes Directed Studies (99 series). Differs from MAPS as UHCC data. Includes Cooperative Education (93 series) as there is a resource cost to Includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

2323

Page 24: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#10#10 Fill RateFill Rate

Total active student registrations in program classes (number of seats filled) Total active student registrations in program classes (number of seats filled) at Fall and Spring census divided by the maximum enrollment (number of at Fall and Spring census divided by the maximum enrollment (number of seats offered).seats offered).

Captured at Census and excludes students who have already withdrawn Captured at Census and excludes students who have already withdrawn (W) at this point.(W) at this point.

2424

Page 25: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#11#11 FTE BOR Appointed FacultyFTE BOR Appointed Faculty

Sum appointments (1.0, 0.5, etc.) of all BOR appointed program faculty Sum appointments (1.0, 0.5, etc.) of all BOR appointed program faculty (excludes lecturers and other non BOR appointees).(excludes lecturers and other non BOR appointees).

Uses the “hiring status” of the faculty member – not the teaching/work load.Uses the “hiring status” of the faculty member – not the teaching/work load.

Uses the Employing Agency Code (EAC) recorded in the Human Resources Uses the Employing Agency Code (EAC) recorded in the Human Resources (HR) database to determine faculty’s program home.(HR) database to determine faculty’s program home.

Data as of 10/31/2011Data as of 10/31/2011

Data provided by UH Human Resources Office.Data provided by UH Human Resources Office.

Click here for the count of BOR Appointed Program Faculty in your program:Click here for the count of BOR Appointed Program Faculty in your program:

2012 BOR Appointed Program Faculty2012 BOR Appointed Program Faculty

2525

Page 26: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#12 #12 Majors to FTE BOR Appointed Faculty Majors to FTE BOR Appointed Faculty

Number of majors (#3) divided by sum appointments (#11) (1.0, 0.5, etc.) of Number of majors (#3) divided by sum appointments (#11) (1.0, 0.5, etc.) of all BOR appointed program faculty. all BOR appointed program faculty.

Data show the number of student majors in the program for each faculty Data show the number of student majors in the program for each faculty member (25 majors to 1 faculty shown as “25”)member (25 majors to 1 faculty shown as “25”)

2626

Page 27: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#13#13 Majors to Analytic FTE FacultyMajors to Analytic FTE Faculty

Number of majors (#3) divided by number of Analytic FTE faculty (13a).Number of majors (#3) divided by number of Analytic FTE faculty (13a).

2727

Page 28: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#13a#13a Analytical FTE Faculty (Workload)Analytical FTE Faculty (Workload)

Calculated by sum of Semester Hours (not Student Semester Hours) taught Calculated by sum of Semester Hours (not Student Semester Hours) taught in program classes divided by 27.in program classes divided by 27.

Analytic FTE is useful as a comparison to FTE of BOR appointed faculty Analytic FTE is useful as a comparison to FTE of BOR appointed faculty (#11). Used for analysis of program offerings covered by lecturers.(#11). Used for analysis of program offerings covered by lecturers.

2828

Page 29: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#14#14 OverallOverall Program Budget AllocationProgram Budget Allocation

The overall program budget allocation = General Funded Budget Allocations The overall program budget allocation = General Funded Budget Allocations (14a) + Special/Federal Budget Allocations (14b) + Tuition and Fees (14c) (14a) + Special/Federal Budget Allocations (14b) + Tuition and Fees (14c) + Other fees+ Other fees

The overall program budget allocation The overall program budget allocation is automatically calculated when you is automatically calculated when you enter your general funded budget allocation, special/federal budget enter your general funded budget allocation, special/federal budget allocation, other funds, or tuition and fees, into the online tool tab called, allocation, other funds, or tuition and fees, into the online tool tab called, “Cost per SSH.” Again, Joni will upload the data for you this year.“Cost per SSH.” Again, Joni will upload the data for you this year.

The overall program budget allocation is to be determined by the College The overall program budget allocation is to be determined by the College and it includes: Salaries (general funds, special funds, etc.), overload, and it includes: Salaries (general funds, special funds, etc.), overload, lecturers, costs for all faculty and staff assigned to the program, supply and lecturers, costs for all faculty and staff assigned to the program, supply and maintenance, and tuition and fees.maintenance, and tuition and fees.

2929

Page 30: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#14a#14a General Funded Budget AllocationGeneral Funded Budget Allocation

The general funded budget allocation = actual personnel costs + b budget The general funded budget allocation = actual personnel costs + b budget expendituresexpenditures

3030

Page 31: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#14b#14b Special/Federal Budget AllocationSpecial/Federal Budget Allocation

The expenditure of dollars from Federal grantsThe expenditure of dollars from Federal grants

3131

Page 32: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#14c#14c Tuition and FeesTuition and Fees

New this year to be included in your programs overall budget allocation is New this year to be included in your programs overall budget allocation is the amount collected for tuition and fees in the 2012 academic year.the amount collected for tuition and fees in the 2012 academic year.

3232

Page 33: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#15#15 Cost per SSHCost per SSH

Overall Program Budget Allocation (#14) divided by SSH in all program Overall Program Budget Allocation (#14) divided by SSH in all program classes (#6)classes (#6)

This value is automatically calculated for you when you enter your general This value is automatically calculated for you when you enter your general funded budget allocation, special/federal budget allocation, other funds, or funded budget allocation, special/federal budget allocation, other funds, or tuition and fees, into the online tool tab called, “Cost per SSH.”tuition and fees, into the online tool tab called, “Cost per SSH.”

3333

Page 34: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#16#16 Number of Low Enrolled (<10) ClassesNumber of Low Enrolled (<10) Classes

Classes taught (#8) with 9 or fewer active students at Census. Classes taught (#8) with 9 or fewer active students at Census.

Excludes students who have already withdrawn (W) at this point.Excludes students who have already withdrawn (W) at this point.

Excludes Directed Studies (99 series).Excludes Directed Studies (99 series).

Includes Cooperative Education (93 series) as there is a resource cost to Includes Cooperative Education (93 series) as there is a resource cost to the program.the program.

3434

Page 35: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on efficiencyefficiency

This year the system office will calculate and report health calls for all This year the system office will calculate and report health calls for all instructional programs using AY 2012 data. The following instructions instructional programs using AY 2012 data. The following instructions illustrate how those calls are made.illustrate how those calls are made.

Program Efficiency is calculated using 2 separate measures…Fill rate Program Efficiency is calculated using 2 separate measures…Fill rate (#10), and Majors to FTE BOR Appointed Faculty (#12). (#10), and Majors to FTE BOR Appointed Faculty (#12).

The following benchmarks are used to determine health for Fill Rate:The following benchmarks are used to determine health for Fill Rate:

Healthy:Healthy: 75 – 100%75 – 100%Cautionary:Cautionary: 60 – 74%60 – 74%Unhealthy:Unhealthy: < 60%< 60%

An Overall Category Health Score is assigned where:An Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1 = Cautionary1 = Cautionary0 = Unhealthy0 = Unhealthy

3535

Page 36: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on efficiencyefficiency cont… cont…

The following benchmarks are used to determine health for Majors/FTE BOR Appointed The following benchmarks are used to determine health for Majors/FTE BOR Appointed Faculty :Faculty :

Healthy:Healthy: 15 - 3515 - 35Cautionary:Cautionary: 30 – 60; 7 - 1430 – 60; 7 - 14Unhealthy:Unhealthy: 61 +; 6 or fewer61 +; 6 or fewer

An Overall Category Health Score is assigned where:An Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1 = Cautionary1 = Cautionary0 = Unhealthy0 = Unhealthy

Finally, average the 2 overall health scores for Class fill rate and Majors/FTE BOR Finally, average the 2 overall health scores for Class fill rate and Majors/FTE BOR Appointed Faculty then use the following rubric:Appointed Faculty then use the following rubric:

1.5 - 2 1.5 - 2 = Healthy= Healthy.5 - 1.5 - 1 = Cautionary= Cautionary00 = Unhealthy= Unhealthy

3636

Page 37: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#17#17 Successful Completion Successful Completion (Equivalent C or higher) (Equivalent C or higher)

Percentage of students actively enrolled in program classes at Fall and Percentage of students actively enrolled in program classes at Fall and Spring census who at end of semester have earned a grade equivalent to C Spring census who at end of semester have earned a grade equivalent to C or higher.or higher.

3737

Page 38: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#18#18 Withdrawals (grade = W)Withdrawals (grade = W)

Number of students actively enrolled (at this point have not withdrawn) at Number of students actively enrolled (at this point have not withdrawn) at Fall and Spring census who at end of semester have a grade of W.Fall and Spring census who at end of semester have a grade of W.

3838

Page 39: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#19#19 Persistence Fall to SpringPersistence Fall to Spring

Count of students who are majors in program at fall census (from Fall Count of students who are majors in program at fall census (from Fall semester #3) and at subsequent Spring semester census are enrolled and semester #3) and at subsequent Spring semester census are enrolled and are still majors in the program. are still majors in the program.

Example: Example: 31 majors start in Fall31 majors start in Fall

21 majors of the original 31 persist into Spring21 majors of the original 31 persist into Spring21/31 = .6774 or 67.74%21/31 = .6774 or 67.74%

3939

Page 40: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#20#20 Unduplicated Degrees/Certificates AwardedUnduplicated Degrees/Certificates Awarded

Unduplicated headcount of students in the fiscal year reported to whom a program Unduplicated headcount of students in the fiscal year reported to whom a program degree or any certificate has been conferred. (Sum of 20a, 20b, 20c, and 20d). degree or any certificate has been conferred. (Sum of 20a, 20b, 20c, and 20d).

[sum of 20a, 20b, 20c, and 20d, then unduplicated] [sum of 20a, 20b, 20c, and 20d, then unduplicated]

Uses most recent available freeze of fiscal year data.Uses most recent available freeze of fiscal year data.

For ARPD year 2010, the most recent fiscal data on August 15, 2010, was from FY For ARPD year 2010, the most recent fiscal data on August 15, 2010, was from FY 2010. 2010. 

For ARPD year 2011, the most recent fiscal data on August 15, 2011, was from FY For ARPD year 2011, the most recent fiscal data on August 15, 2011, was from FY 2011.2011.

For ARPD year 2012, the most recent fiscal data on August 15, 2012, was from FY For ARPD year 2012, the most recent fiscal data on August 15, 2012, was from FY 2012.2012.

4040

Page 41: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#20a#20a Number of Degrees AwardedNumber of Degrees Awarded

Degrees conferred in the FISCAL_YEAR_IRO.Degrees conferred in the FISCAL_YEAR_IRO.

The count is of degrees and may show duplicate degrees received in the The count is of degrees and may show duplicate degrees received in the program by the same student if the program offers more than one degree.program by the same student if the program offers more than one degree.

Uses most recent available freeze of fiscal year data. Uses most recent available freeze of fiscal year data.

FISCAL_YEAR_IRO:FISCAL_YEAR_IRO:

“ “Fiscal year, where the value indicates the ending of the fiscal year. For Fiscal year, where the value indicates the ending of the fiscal year. For example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004‐example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004‐2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2004, and Spring 2005 semesters…”2004, and Spring 2005 semesters…”

4141

Page 42: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#20b#20b Certificates of Achievement AwardedCertificates of Achievement Awarded

Certificates of achievement conferred in the FISCAL_YEAR_IRO.Certificates of achievement conferred in the FISCAL_YEAR_IRO.

The count is of program certificates of achievement and may show multiple The count is of program certificates of achievement and may show multiple certificates of achievement in the same program received by the same certificates of achievement in the same program received by the same student.student.

Uses most recent available freeze of fiscal year data.Uses most recent available freeze of fiscal year data.

FISCAL_YEAR_IRO:FISCAL_YEAR_IRO:

“ “Fiscal year, where the value indicates the ending of the fiscal year. For Fiscal year, where the value indicates the ending of the fiscal year. For example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004‐example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004‐2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2004, and Spring 2005 semesters…”2004, and Spring 2005 semesters…”

4242

Page 43: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#20c#20c Advanced Professional Certificates AwardedAdvanced Professional Certificates Awarded

The count is of program Advanced Professional Certificates and may show The count is of program Advanced Professional Certificates and may show multiple Advanced Professional Certificates in the same program received multiple Advanced Professional Certificates in the same program received by the student.by the student.

Uses most recent available freeze of fiscal year data. Uses most recent available freeze of fiscal year data.

4343

Page 44: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#20d#20d OtherOther Certificates AwardedCertificates Awarded

The count is of other program certificates (such as the Academic Subject The count is of other program certificates (such as the Academic Subject Certificate) and will show multiples received by the same student.Certificate) and will show multiples received by the same student.

Uses most recent available freeze of fiscal year data. Uses most recent available freeze of fiscal year data.

4444

Page 45: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#21#21 External Licensing Exams PassedExternal Licensing Exams Passed

These are data to be entered by the college (the same information currently These are data to be entered by the college (the same information currently collected on the Graduates and Leavers surveys). Note: Not all CTE collected on the Graduates and Leavers surveys). Note: Not all CTE programs have external licensing exams.programs have external licensing exams.

This value will be automatically calculated when you click on the “External” This value will be automatically calculated when you click on the “External” tab within the online tool web submission. Just click the edit button for your tab within the online tool web submission. Just click the edit button for your program, then enter the number sitting for the exam and the number that program, then enter the number sitting for the exam and the number that passed. Then just click on the button, “Save External Data.”passed. Then just click on the button, “Save External Data.”

4545

Page 46: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#22#22 Transfers to UH 4-yr programsTransfers to UH 4-yr programs

Students with home campus UH Manoa, UH Hilo, or UH West Oahu for the Students with home campus UH Manoa, UH Hilo, or UH West Oahu for the first time in the reporting Fall who prior to the reporting Fall had UH first time in the reporting Fall who prior to the reporting Fall had UH community college as home campus.community college as home campus.

Also includes students who for the first time in the reporting Fall had Maui Also includes students who for the first time in the reporting Fall had Maui College as home campus and major ABIT or ENGT.College as home campus and major ABIT or ENGT.

This is a program measure. A student is included in the count of program This is a program measure. A student is included in the count of program transfers in as many programs in which they have been a major at the transfers in as many programs in which they have been a major at the college.college.

4646

Page 47: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#23#23 Transfers with credential from programTransfers with credential from program

Students included in #22 who have received a degree from the community Students included in #22 who have received a degree from the community college program prior to transfer. college program prior to transfer.

Does not include any certificates. Does not include any certificates.

4747

Page 48: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#22b#22b Transfers without credential from programTransfers without credential from program

Students included in #22 who did not receive a degree from the community Students included in #22 who did not receive a degree from the community college program prior to transfer.college program prior to transfer.

4848

Page 49: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on effectivenesseffectiveness

This year the system office will calculate and report health calls for all instructional This year the system office will calculate and report health calls for all instructional programs using academic year 2012 data. The following instructions illustrate how programs using academic year 2012 data. The following instructions illustrate how those calls are made.those calls are made.

Program Effectiveness is calculated using 3 separate measures: Unduplicated Program Effectiveness is calculated using 3 separate measures: Unduplicated Degrees/Certificates Awarded (#20) / Majors (#3), Unduplicated Degrees/Certificates Degrees/Certificates Awarded (#20) / Majors (#3), Unduplicated Degrees/Certificates Awarded (#20) / Annual new and replacement positions (County prorated) (#2), and Awarded (#20) / Annual new and replacement positions (County prorated) (#2), and Persistence Fall to Spring (#19).Persistence Fall to Spring (#19).

The following benchmarks are used to determine health for Unduplicated The following benchmarks are used to determine health for Unduplicated Degrees/Certificates Awarded per major:Degrees/Certificates Awarded per major:

Healthy:Healthy: > 20%> 20%Cautionary:Cautionary: 15 - 20%15 - 20%Unhealthy:Unhealthy: < 15%< 15%

An Overall Category Health Score is assigned where:An Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1= Cautionary1= Cautionary0= Unhealthy0= Unhealthy

4949

Page 50: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on effectivenesseffectiveness cont… cont…

The second measure used to determine health is Unduplicated Degrees/Certificates The second measure used to determine health is Unduplicated Degrees/Certificates Awarded (#20) / Annual new and replacement positions (County prorated) (#2).Awarded (#20) / Annual new and replacement positions (County prorated) (#2).

The following benchmarks are used to for this measure:The following benchmarks are used to for this measure:

Healthy:Healthy: .75 – 1.5.75 – 1.5Cautionary:Cautionary: .25 - .75 and 1.5 – 3.0.25 - .75 and 1.5 – 3.0Unhealthy:Unhealthy: < .25 and >3.0< .25 and >3.0

An Overall Category Health Score is assigned where:An Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1 = Cautionary1 = Cautionary0 = Unhealthy0 = Unhealthy

5050

Page 51: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on effectivenesseffectiveness cont… cont…

The third measure used to determine health is Persistence (Fall to Spring) (#19).The third measure used to determine health is Persistence (Fall to Spring) (#19).

The following benchmarks are used for this measure:The following benchmarks are used for this measure:

Healthy:Healthy: 75 - 100%75 - 100%Cautionary:Cautionary: 60 - 74%60 - 74%Unhealthy:Unhealthy: < 60%< 60%

An Overall Category Health Score is assigned where:An Overall Category Health Score is assigned where:

2 = Healthy2 = Healthy1 = Cautionary1 = Cautionary0 = Unhealthy0 = Unhealthy

5151

Page 52: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s health based on Determination of program’s health based on effectivenesseffectiveness cont… cont…

You should now have a value of zero, one, or two for each of the 3 effectiveness You should now have a value of zero, one, or two for each of the 3 effectiveness measures. The process of determining the Effectiveness health call score contains the measures. The process of determining the Effectiveness health call score contains the following 3 steps:following 3 steps:

Step #1: Add up all 3 Overall Category Health scores for the effectiveness measures. Step #1: Add up all 3 Overall Category Health scores for the effectiveness measures. (the zero’s, one’s and two's you assigned earlier)(the zero’s, one’s and two's you assigned earlier)

Step #2: Determine the effectiveness category health call range where:Step #2: Determine the effectiveness category health call range where:

5 - 6 5 - 6 = Healthy= Healthy2 - 4 2 - 4 = Cautionary= Cautionary0 - 1 0 - 1 = Unhealthy= Unhealthy

Step #3: Now use the scoring rubric below to determine the effectiveness health call Step #3: Now use the scoring rubric below to determine the effectiveness health call score: (for example: you had a healthy 5 in the previous step you would assign it a score: (for example: you had a healthy 5 in the previous step you would assign it a healthy 2 here)healthy 2 here)

2 2 = Healthy= Healthy1 1 = Cautionary= Cautionary0 0 = Unhealthy= Unhealthy

5252

Page 53: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Determination of program’s Determination of program’s overalloverall health health

You should now have a value of zero, one, or two for each of the 3 program health You should now have a value of zero, one, or two for each of the 3 program health calls; Demand, Efficiency, and Effectiveness. Simply add those 3 values together calls; Demand, Efficiency, and Effectiveness. Simply add those 3 values together and use the Scoring Range Rubric below to determine the overall health of your and use the Scoring Range Rubric below to determine the overall health of your program.program.

5 - 6 5 - 6 = Healthy= Healthy2 - 4 2 - 4 = Cautionary= Cautionary0 - 1 0 - 1 = Unhealthy= Unhealthy

5353

Page 54: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#23#23 Number of Distance Education Classes TaughtNumber of Distance Education Classes Taught

Measures the number of classes taught with the mode of delivery as Measures the number of classes taught with the mode of delivery as “Distance Completely Online.”“Distance Completely Online.”

In setting up the class, the college indicates the method of instruction used In setting up the class, the college indicates the method of instruction used by the instructor in conducting the class. by the instructor in conducting the class.

If the method is Distance Education, and the college indicates the “mode” of If the method is Distance Education, and the college indicates the “mode” of distance delivery as “Distance Completely Online” the class will be included distance delivery as “Distance Completely Online” the class will be included in this count.in this count.

5454

Page 55: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#24#24 Enrollment Distance Education ClassesEnrollment Distance Education Classes

At the Fall and Spring census, the number of students actively enrolled in all At the Fall and Spring census, the number of students actively enrolled in all classes owned by the program and identified as Distance Completely On‐classes owned by the program and identified as Distance Completely On‐Line (#23).Line (#23).

Does not include students who at Census have already withdrawn from the Does not include students who at Census have already withdrawn from the class.class.

5555

Page 56: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#25#25 (DE) Fill Rate(DE) Fill Rate

Total active student registrations in program distance education classes Total active student registrations in program distance education classes (#24) classes (number of seats filled) at Fall and Spring census divided by (#24) classes (number of seats filled) at Fall and Spring census divided by the maximum enrollment (number of seats offered). the maximum enrollment (number of seats offered).

Does not include students who at Census have already withdrawn from the Does not include students who at Census have already withdrawn from the class.class.

5656

Page 57: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#26#26 (DE) Successful Completion (Equivalent C or higher)(DE) Successful Completion (Equivalent C or higher)

Percentage of students enrolled in program Distance Education classes Percentage of students enrolled in program Distance Education classes (#24) at Fall and Spring census who at end of semester have earned a (#24) at Fall and Spring census who at end of semester have earned a grade equivalent to C or higher.grade equivalent to C or higher.

5757

Page 58: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#27#27 (DE) Withdrawals (Grade=W) (DE) Withdrawals (Grade=W)

Number of students actively enrolled in program Distance Education Number of students actively enrolled in program Distance Education classes (#24) at Fall and Spring census who at end of semester classes (#24) at Fall and Spring census who at end of semester have a grade of W.have a grade of W.

5858

Page 59: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#28#28 (DE) Persistence (Fall to Spring not limited to Distance Ed) (DE) Persistence (Fall to Spring not limited to Distance Ed)

Students enrolled in program distance education classes at Fall census who Students enrolled in program distance education classes at Fall census who at subsequent Spring semester census are enrolled in the college.at subsequent Spring semester census are enrolled in the college.

Not limited to students continuing to take program distance education Not limited to students continuing to take program distance education classes.classes.

Example: Example: 31 students enrolled in DE classes in Fall31 students enrolled in DE classes in Fall

21 students of the original 31 students persist into any Spring class21 students of the original 31 students persist into any Spring class21/31 = .6774 or 67.74%21/31 = .6774 or 67.74%

5959

Page 60: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#29#29 Perkins Core Indicator: Technical Skills Attainment Perkins Core Indicator: Technical Skills Attainment (1P1)(1P1)

Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. reports completed annually for CTE programs. All Perkins Core Indicator goals not met must be All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your review.addressed in the narrative and action plan in your review.

A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins year.and who has completed 12 or more credits by the end of the Perkins year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.and College actual.

Technical Skills Attainment is calculated by:Technical Skills Attainment is calculated by:

    Number of concentrators who have a cumulative GPA> or =Number of concentrators who have a cumulative GPA> or = 2.00 in Career and Technical Education courses and who have stopped 2.00 in Career and Technical Education courses and who have stopped program participation in the year reported.program participation in the year reported.

Number of concentrators who have stopped program participation in the year reported.Number of concentrators who have stopped program participation in the year reported.

6060

Page 61: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#30#30 Perkins Core Indicator: Completion (2P1)Perkins Core Indicator: Completion (2P1)

Perkins Core Indicators are used in the development of the HawCC Program Health Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. Indicator (PHI) reports completed annually for CTE programs. All Perkins Core All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your Indicator goals not met must be addressed in the narrative and action plan in your review.review.

A concentrator is a student who has a major (taken from the major field in Banner) for A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins a CTE program, and who has completed 12 or more credits by the end of the Perkins year.year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.show State goal and College actual.

Completion is calculated by:Completion is calculated by:

Number of concentrators who received a degree or certificate in a Career and Technical Education program and who Number of concentrators who received a degree or certificate in a Career and Technical Education program and who have stopped program participation in the year reported.have stopped program participation in the year reported.

Number of concentrators who have stopped program participation in the year reported.Number of concentrators who have stopped program participation in the year reported.

6161

Page 62: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#31#31 Perkins Core Indicator: Student Retention or Transfer Perkins Core Indicator: Student Retention or Transfer (3P1)(3P1)

Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. reports completed annually for CTE programs. All Perkins Core Indicator goals not met must be All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your review.addressed in the narrative and action plan in your review.

A concentrator is a student who has a major (taken from the major field in Banner) for a CTE A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins year.program, and who has completed 12 or more credits by the end of the Perkins year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.and College actual.

Student Retention or Transfer is calculated by:Student Retention or Transfer is calculated by:

Number of concentrators in the year reported who have not completed a program and who continue postsecondary enrollment or who Number of concentrators in the year reported who have not completed a program and who continue postsecondary enrollment or who have transferred to a baccalaureate degree program.have transferred to a baccalaureate degree program.

Number of concentrators in the year reported who have not completed a program.Number of concentrators in the year reported who have not completed a program.

6262

Page 63: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#32#32 Perkins Core Indicator: Student Placement (4P1)Perkins Core Indicator: Student Placement (4P1)

Perkins Core Indicators are used in the development of the HawCC Program Health Indicator Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. (PHI) reports completed annually for CTE programs. All Perkins Core Indicator goals not met All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your review.must be addressed in the narrative and action plan in your review.

A concentrator is a student who has a major (taken from the major field in Banner) for a CTE A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins year.program, and who has completed 12 or more credits by the end of the Perkins year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.goal and College actual.

Student Placement is calculated by:Student Placement is calculated by:

Number of concentrators in the year reported (previous Perkins year) who have stopped program participation and who are placed Number of concentrators in the year reported (previous Perkins year) who have stopped program participation and who are placed or retained in employment, military service, or an apprenticeship program within or retained in employment, military service, or an apprenticeship program within

unemployment insurance quarter following program completion.unemployment insurance quarter following program completion.

Number of concentrators in the year reported (previous Perkins year) who have stopped program participationNumber of concentrators in the year reported (previous Perkins year) who have stopped program participation

6363

Page 64: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#33#33 Perkins Core Indicator: Nontraditional Participation Perkins Core Indicator: Nontraditional Participation (5P1)(5P1)

Perkins Core Indicators are used in the development of the HawCC Program Health Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. Indicator (PHI) reports completed annually for CTE programs. All Perkins Core Indicator All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your review.goals not met must be addressed in the narrative and action plan in your review.

A concentrator is a student who has a major (taken from the major field in Banner) for a A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins year.CTE program, and who has completed 12 or more credits by the end of the Perkins year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.State goal and College actual.

Nontraditional Participation is calculated by:Nontraditional Participation is calculated by:

Number of participants from underrepresented gender groups who participated in a program that leads to employment in Number of participants from underrepresented gender groups who participated in a program that leads to employment in nontraditional fields.nontraditional fields.

Number of participants who participated in a program that leads to employment in nontraditional fields.Number of participants who participated in a program that leads to employment in nontraditional fields.

6464

Page 65: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

#34#34 Perkins Core Indicator: Non-Traditional Completion Perkins Core Indicator: Non-Traditional Completion (5P2)(5P2)

Perkins Core Indicators are used in the development of the HawCC Program Health Indicator Perkins Core Indicators are used in the development of the HawCC Program Health Indicator (PHI) reports completed annually for CTE programs. (PHI) reports completed annually for CTE programs. All Perkins Core Indicator goals not met All Perkins Core Indicator goals not met must be addressed in the narrative and action plan in your review.must be addressed in the narrative and action plan in your review.

A concentrator is a student who has a major (taken from the major field in Banner) for a CTE A concentrator is a student who has a major (taken from the major field in Banner) for a CTE program, and who has completed 12 or more credits by the end of the Perkins year.program, and who has completed 12 or more credits by the end of the Perkins year.

Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State Uses Perkins data from prior year Perkins Consolidated Annual Report (CAR). Data show State goal and College actual.goal and College actual.

Non-Traditional Completion is calculated by:Non-Traditional Completion is calculated by:

Number of concentrators from underrepresented gender groups who received a degree or certificate in a program that leads to Number of concentrators from underrepresented gender groups who received a degree or certificate in a program that leads to employment in nontraditional fields. employment in nontraditional fields.

Number of concentrators who received a degree or certificate in a program that leads to employment in nontraditional fields.Number of concentrators who received a degree or certificate in a program that leads to employment in nontraditional fields.

6565

Page 66: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Comprehensive & Annual Program Review TimelineComprehensive & Annual Program Review Timeline

Develop training materials for Instructional CTE programs

Provide Program Review Training to CTE Instructional Programs

Develop and publish BOR Appointed Faculty and Classes Taught aligned to program files

Publish all Comprehensive CTE Program Reviews to Assessment Website

All Instructional CTE program reviews (Comprehensive and Annual) due to Interim VC for Academic Affairs/ UHCC ARPD website by EOB Friday November 30th.

Follow up on last year’s list of suggested improvements for completion

Edit instructional CTE glossary and health call scoring rubric and provide suggested changes to system office

Plan this year’s program review based on suggested improvements from last year’s review

Evaluate ARPD online web submission site for functionality and report bugs to system office

Administer PR Process Improvement Sessions (Develop questionnaire, schedule sessions, collect feedback)

Work with AC and Interim VCAA to determine what feedback needs to be taken back to UHCC IPRC

Update documentation on website needed for program review

Sept

Sept OctAug-SeptJan 27th

Dec 20th

Jan 25th

Sept

SeptAug Nov 30Sept-Oct

6666

Send Campus-wide update on program review process with link to data and due date

Sept

Summarize PR Process Improvement Feedback, communicate results to groups, publish to web.

Jan 5th

Page 67: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

AY 2012 Comprehensive & Annual Program Review ProcessAY 2012 Comprehensive & Annual Program Review Process

Step 1Step 1 Write your program review using the appropriate template or Write your program review using the appropriate template or online submission tool.online submission tool.

Step 2Step 2 Send yourSend your documents documents (one Word document per review) (one Word document per review) to to Interim VC for Academic Affairs Joni Onishi by email no later Interim VC for Academic Affairs Joni Onishi by email no later than end of business, Friday November 30th, 2012. than end of business, Friday November 30th, 2012.

Step 3Step 3 Interim VCAA will ensure that all required documents Interim VCAA will ensure that all required documents have been received and that they are adequate.have been received and that they are adequate.

Step 4Step 4 Interim VCAA will forward all approved reviews to the Interim VCAA will forward all approved reviews to the Institutional Research Office for further processing.Institutional Research Office for further processing.

Step 5Step 5 The online Annual reviews will be reviewed by the UH Board of The online Annual reviews will be reviewed by the UH Board of Regents. Comprehensive reviews will be forwarded along as Regents. Comprehensive reviews will be forwarded along as appropriate following CERC guidelines.appropriate following CERC guidelines.

Step 6Step 6 All comprehensive reviews will finally be converted to PDF and All comprehensive reviews will finally be converted to PDF and published to the Program Review website.published to the Program Review website.

6767

Page 68: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

Questions?Questions?The intention of this presentation was to provide a single source for all of the The intention of this presentation was to provide a single source for all of the documentation related to the Comprehensive & Annual Program Review documentation related to the Comprehensive & Annual Program Review process—specifically for Instructional CTE programs. All of the documents process—specifically for Instructional CTE programs. All of the documents you should need for your review have been linked directly into this you should need for your review have been linked directly into this presentation.presentation.

If you need more information on this process please feel free to contact me:If you need more information on this process please feel free to contact me:

HawCC Institutional Analyst HawCC Institutional Analyst Shawn FloodShawn Flood 934-2648934-2648

Mahalo!Mahalo!

6868

Page 69: AY 2012 CTE Instructional Program Review Process Data Description Process – Timeline Rev. 10-15-12

CERC Comprehensive Program CERC Comprehensive Program Review Template & ProcessReview Template & Process

James will now discuss the comprehensive instructional program review template, James will now discuss the comprehensive instructional program review template, and briefly cover the current CERC process.and briefly cover the current CERC process.

Comprehensive Instruction Program Review TemplateComprehensive Instruction Program Review Template

If you need assistance please contact me:If you need assistance please contact me:

James KileyJames KileyInstitutional Assessment CoordinatorInstitutional Assessment Coordinator Email: [email protected] (Email is best)Email: [email protected] (Email is best)Phone: 934-2649Phone: 934-2649

Mahalo!Mahalo!

6969