North Carolina CTE Program Eval Instrumentschd.ws/hosted_files/2017springmeetingagenda/88/North... · Web viewProgram Evaluation Instrument Career and Technical Education Guide for

Career and Technical EducationProgram Evaluation Instrument

Career and Technical Education Guide for CTE Program Evaluation

Public Schools of North Carolina

State Board of Education Department of Public Instruction

Career and Technical Education Division

Raleigh, North Carolina

July 1, 2016

Rationale for CTE Program Evaluation Instrument

The development and utilization of the CTE Program Evaluation Instrument is an effort to improve Perkins IV accountability systems. The Carl D. Perkins Career and Technical Education Improvement Act of 2006 (commonly called the Perkins Act) provides federal funding and regulatory authority for Career and Technical Education. The following sections of the Perkins Act provide guidance specific to evaluation, improvement, and effectiveness of CTE programs.

...interested individuals are involved in the development, implementation, and evaluation of career and technical education programs assisted under this title, and how such individuals and entities are effectively informed about, and assisted in understanding, the requirements of this title, including career and technical programs of study. (Section 135 b. (5))

Provide assurances that the eligible recipient will provide a career and technical education program that is of such size, scope, and quality to bring about improvement in the quality of career and technical education programs. (Section 135 b. (6))

Describe the process that will be used to evaluate and continuously improve the performance of the eligible recipient. (Section 135 b. (7))

Provide services and activities that are of sufficient size, scope, and quality to be effective. (Section 135 b. (8))

The North Carolina General Assembly provides regulatory authority specific to Career and Technical Education via General Statute 115c Article 10. The following sections of the State Statutes provide guidance specific to evaluation and effectiveness of CTE programs.

The LEA has on file the findings of evaluations of career and technical education programs operated in the LEA during the previous one year and five years to ascertain the effectiveness of instruction, services, and activities. (Section 115C-154(10))

An organized system for conducting follow-up studies to determine the effectiveness of the career and technical education programs and guidance and counseling is used by the principal, and a compilation of all schools maintained in the office of the career and technical education director. (Section 115C-154(10))

In compliance with federal law, the NC Department of Public Instruction administers all state-operated educational programs, employment activities and admissions without discrimination because of race, religion, national or ethnic origin, color, age, military service, disability, or gender, except where exemption is appropriate and allowed by law.

Purpose of CTE Program Evaluation Instrument

The CTE Program Evaluation Instrument should be utilized as an additional resource to analyze and evaluate CTE performance and effectiveness at the local level. The instrument is not intended to measure all standards of quality but rather key data that would inform success. This instrument may also be customized at the local level to include additional standards and indicators. The results of the instrument are not intended to be utilized for regional or state comparison purposes but rather for LEA improvement.

Suggested CTE Program Evaluation Implementation

Initially, the instrument should be utilized to collect benchmark data. The data entered into the Local Benchmark and Local Data columns should match and will become the original baseline data for future comparison. During subsequent evaluations, the Local Benchmark data should reflect data from the previous evaluation and the Local Data information should reflect the most current data available. Once the benchmark and local data is entered, the Rating, Value, and Total columns may also be determined.

Upon completion of the CTE program evaluation, identify overall standard and individual indicator ratings to determine areas of strength and opportunities for growth. Develop a continuous improvement plan for the identified standard/indicator with measurable outcomes and attainable goals. Monitor, measure and identify progress of continuous improvement on a regular basis. The information developed will serve as an additional resource as the local plan in the local planning system (LPS) is developed and revised annually.

Suggested CTE Program Evaluation Instrument Delivery Methods

District CTE Program EvaluationThe CTE Program Evaluation Instrument is used to analyze and evaluate district CTE performance and effectiveness. The data utilized represent the CTE program for the district.

District CTE Program Area EvaluationThe CTE Program Evaluation Instrument is used to analyze CTE performance and effectiveness in one or more of the eight CTE program areas offered throughout the district. The data utilized represent the specific CTE program area identified.

School CTE Program EvaluationThe CTE Program Evaluation Instrument is used to analyze school CTE performance and effectiveness. The data utilized represent the CTE program of the specific school identified.

North Carolina CTE Program Quality Standards

Standard 1: Student Engagement

All students are eligible and encouraged to be served by a career and technical education program regardless of race, religion, national or ethnic origin, socioeconomic status, color, age, disability, or gender. Continual guidance and advising is provided to ensure availability of CTE opportunities for student engagement.

Standard 2: Facilities and Equipment

Facilities and equipment must support CTE instruction of technical and academic content standards and reflect current and emerging technologies. Facilities include classroom, inside and outside labs, instructor work area, display area, shop area, greenhouse, inside and outside storage area. Safety is of utmost importance and facilities and equipment must be continuously evaluated.

Standard 3: Technical Skills Assessment

National, state, and/or local assessments provide ongoing information on the extent to which students are attaining the necessary knowledge and skills for entry into and advancement in post-secondary education and careers in their chosen program of study.

Standard 4: Career Development

Career development and advisement help students make informed decisions about which CTE program of study to pursue. Information and tools should be provided to help students learn about Career Clusters, Career Pathways, CTSOs, work-based learning, post-secondary, and credentialing opportunities.

Standard 5: High School Completion

CTE concentrators are provided a rigorous program of study while following a coherent sequence of courses aligned to the student’s post-secondary goals. The completed program of study provides a seamless transition to post-secondary education and/or the workforce.

Standard 6: Postsecondary Preparation

Preparation includes a systematic, seamless process for students to earn college credit for post-secondary courses taken in high school (Career and College Promise), transfer high school credit to post-secondary institutions (Local and State Articulation Agreement), and transfer credit earned at a two-year college to any four-year institution (Comprehensive Articulation Agreement).

Standard 7: Connection with Business and Industry

Ongoing relationships among education, business, and other community stakeholders must occur in the CTE program design, implementation, and maintenance. Collaborative partnerships should conduct ongoing analyses of economic and workforce trends to identify local, regional, and statewide needs. Analysis of needs assist in the creation, expansion, or redirection of existing CTE programs. Partnerships should consider linking into existing initiatives that promote workforce and economic development, such as sector strategies and the development of Career Pathways.

Standard 8: Curriculum and Program Design

Content standards that define what students are expected to know and be able to do to enter and advance in college and/or their career comprise the foundation of a CTE program of study. Rigorous college and career readiness standards should incorporate essential knowledge and skills which students must master regardless of their chosen career area. Standards must be developed and continually validated in collaboration with secondary, post-secondary, and industry partners. A coherent sequence of secondary and/or post-secondary courses must be developed to ensure successful student transition to post-secondary education without duplicating courses or requiring remedial coursework.

CTE Program Evaluation Instrument Definitions

Indicator Quality to be measured to determine the effectiveness of the identified standard

Measure Method to determine a quantifiable or qualitative result

Data Source Suggested resources to utilize to determine the measure

Local Benchmark Measure previously determined from the prior review or year

Local Data Current measure determined from the most recent data available

Rating Provides an opportunity to compare the current measure to the previous review or year measure; the appropriate selection is made and the comparison value is entered.

Value Predetermined amount representing the rating selection per indicator

Total Value determined during the current review representing the rating selection per indicator

Standard Total Total of each value determined during the current review representing an overall rating of the standard

Overall Total Total value determined during the current review representing an overall rating of all eight standards

Standard 1: Student Engagement

Indicator Measure Suggested Data Sources

Local Benchmark

Local Data Rating Value Total

1.1 - CTE enrollmentsLEA may consider expanding this section per program area. Each program area would receive an individual rating.

Number

PowerSchool Local Planning

System FILR

☐ Increased by ________ ☐ No change☐ Decreased by ________

210

1.2 - CTE concentratorsLEA may consider expanding per Cluster area. Each Cluster area would receive an individual rating.

Number PowerSchool☐ Increased by ________ ☐ No change☐ Decreased by ________

210

1.3 - CTE concentrators among all high school graduates

Percent (concentrators/graduates)

PowerSchool ☐ Increased by ________ ☐ No change☐ Decreased by ________

210

1.4 - CTE enrollments from underrepresented groups (non-traditional by gender, underrepresented racial/ethnic groups)LEA may consider expanding per sub-group. Each sub-group would receive an individual rating.

Percent (students from under-represented groups /CTE enrollments)

PowerSchool LPS


210

1.5 - CTE enrollments in Career and Technical Student Organizations (CTSOs) LEA may consider expanding per CTSO. Each CTSO would receive an individual rating.

Percent (CTSO participants/ CTE enrollments)

CTSO State and National Membership


210

1.6 - CTE enrollments in work-based learning LEA may consider expanding per WBL (shadowing, internships, apprenticeships, etc). Each WBL would receive an individual rating.

Percent (WBL students/CTE enrollments)

PowerSchool WBL Inventory


210

1.7 - National Technical Honor Society (NTHS) Enrollments Number Membership


210

Standard 1 Total (max. 14 points)

Additional comments:

Standard 2: Facilities and Equipment

Indicator Measure Suggested Data Sources Local Benchmark Local Data Value Total

2.1 - Equipment meets state-level CTE program guidelines

Comparison of program equipment inventory with state standards

Equipment inventories Specification sheet or standards

list

☐ Exceeds☐ Meets☐ Does not meet


210

2.2 - Equipment to student ratio meets state-level guidelines

Comparison of number of students per class to equipment

CTE Equipment Guide☐ Exceeds☐ Meets☐ Does not meet

☐ Exceeds☐ Meets☐Does not meet

210

2.3 - Program equipment is relevant to current industry standards

Comparison of program resources with industry practices in local area and/or state

Consultation with advisory board Equipment inventories



210

2.4 - Equipment is in good working order and meets safety requirements

Review of regulatory standards, safety and equipment

Review of OSHA and fire code standards

Safety inspection reports Safety/injury record audit Maintenance check lists



210

2.5 - Adequate classroom and/or workshop space available

Observation of classroom/CTE Lab

Equipment/course guide Capacity guidelines Safety requirements Facilities Guidelines



210

2.6 - Safety inspection of CTE labs/shop areas are conducted annually (at minimum)

Documented safety inspections with improvement efforts

National Institute for Occupational Safety and Health (NIOSH)



210

2.7 - Instructional facilities are clean, well-organized, and conducive to learning





210



Standard 3: Technical Skills Assessment

Indicator Measure Suggested Data Sources Local Benchmark


3.1 - CTE enrollments achieving technical attainment proficiency

Percent (number met proficiency/95% of validated CTE enrollment)

LPS PowerSchool


210

3.2 - CTE enrollments earning a NCRC or other industry-recognized credential

Number (number earning credential) Credential Report

PowerSchool


210

3.3 - CTE concentrators earning a NCRC or other industry-recognized credential

Percent (number of concentrators earning credential/CTE concentrators)

Credential Report Community College Career

Readiness Certification database

PowerSchool


210

3.4 - Proportion of credentials earned to CTE enrollment

Percent (number of credentials earned/CTE enrollment)

Credential Report☐ Increased by ________ ☐ No change☐ Decreased by ________

210



Standard 4: Career Development


Local Data Ratings Value Total

4.1 - Career Interest Inventories administered

Number of Career Interest Inventories administered

Career Development Coordinator

CFNC or other 3rd party vendor


210

4.2 - CTE enrollee Career Development Plans aligned to career goals

Percent (number of Career Development Plans/CTE enrollment)


CFNC or other 3rd party vendor

☐ 95%-100%☐ 75%-94%☐ Less than 75%

210

4.3 - Career Development Plans are reviewed with all CTE enrollees to provide continual guidance and advisement

Percent (number of Career Development Plans reviewed annually/CTE enrollment)


☐ 95%-100%☐ 75%-94%☐ Less than 75%

210

4.4 - Career development information and opportunities are shared verbally (group or individual) with students, parents, staff and other stakeholders

Number of informational events held annually


☐ 4 or more events☐ 1-3 events☐ Less than 1 event

210

4.5 - Career development information and opportunities are shared through other means of non-verbal media (brochures, websites, flyers, social media, etc.)

Number of non-verbal media methods utilized annually


☐ 4 or more methods☐ 1-3 methods☐ Less than 1 method

210

4.6 - Career development information provided includes the following:

Local, regional and/or state Occupational Needs/Demands

Career Clusters/Pathways

CTSOs

Number of CTE opportunities listed promoted through verbal and/or non-verbal media


☐ Information provided includes all of the opportunities listed

☐ Information provided includes 4 or more of the opportunities listed

☐ Information provided includes less than 4 of the opportunities listed

2

1

0



WBL Credentialing Career and College

Promise Articulation

4.7 - CTE enrollees participate in employer visits and/or career fairs

Percent (number of CTE enrollees participating/ number of CTE enrollees)

Student survey Field trip records Career Development

Coordinator WBL Inventory


210



Standard 5: High School Completion


Local Benchmark


5.1 - CTE concentrators earning a high school diploma in four years (“on time” graduation)

Percent (CTE concentrators graduating on time/CTE concentrators)

PowerSchool LPS


210

5.2 - CTE concentrators earning a career diploma endorsement

Percent (CTE concentrators earning CDE/CTE concentrators)

Diploma analysis report


210

5.3 - CTE concentrators earning a college or college UNC diploma endorsement

Percent (CTE concentrators earning college or UNC endorsement/CTE concentrators)



210

5.4 - CTE concentrators earning a college or college UNC diploma AND career diploma endorsement

Percent (CTE concentrators earning college or UNC endorsement AND career diploma/CTE concentrators)



210



Standard 6: Postsecondary Preparation


Local Benchmark


6.1 - CTE enrollees in courses that offer post-secondary credit (CTE CCP and articulation)

Number (CTE enrollees in CTE courses that offer post-secondary credit)

PowerSchool☐ Increased by ________ ☐ No change☐ Decreased by ________

210

6.2 - Career and College Promise (CCP) credits earned by CTE enrollees

Number (Career and College Promise (CCP) credits earned by CTE enrollees)

Power School CCP records


210

6.3 - Post-secondary articulation credits earned by CTE enrollees

Number (post-secondary articulation credits earned by CTE enrollees)

Power School Articulation

records


210

6.4 - CTE concentrators in higher education that require developmental/remedial education

Ratio (CTE concentrators needing remedial education/ CTE concentrators who enrolled in public institutions of higher education)

Community college data system

Community College Multiple Measures Criteria


210

6.5 - CTE concentrators positively placed one year after graduation (military, workforce and/or post-secondary education)LEA may consider expanding per post-secondary placement. Each post-secondary placement would receive an individual rating.

Percent of students employed and/or in education, military or advanced training/students not employed and/or in education, military or advanced training

Concentrator survey

LPS


210



Standard 7: Connections with Business and Industry


Local Benchmark


7.1 - Advisory council members represent employers in local, regional, or state “Star Jobs” fields

Percent (advisory board member who represent “Star Jobs” fields/ all advisory board members)

Advisory council rosters

NC Department of Labor reports

Local chambers of commerce and workforce development boards

List of Star Jobs at nccommerce.com

☐ Highly representative (more than 50 percent)

☐ Representative (11 to 49 percent)

☐ Not representative (10 percent or less)

2

1

0

7.2 - NCWorks Career Pathways certification achieved by advisory council

Number of Career Pathways certified by NCWorks

NCWorks Certification


210

7.3 - Advisory council members are engaged and participate in the planning and implementation process of the CTE program

Number of documented advisory council outcomes

Advisory council rosters

Advisory council meeting minutes

Advisory council strategic plan

Documented advisory council outcomes


210



Standard 8: Curriculum and Program Design



8.1 - CTE courses offered are aligned with the knowledge and skills needed by local, regional, and/or state industry

Input from business and industry on course content

Course descriptions/syllabi

Advisory council survey or evaluation of course content

☐ Closely aligned☐ Somewhat aligned☐ Poorly aligned

210

8.2 - Career Pathways are based on labor market data and aligned to local, regional, and/or state needs.

Number of Career Pathways that are labor market driven

Course guides/offerings NC Department of Labor

reports Local chambers of commerce

and workforce development boards

List of hot jobs at nccommerce.com

☐ Increased by ________☐ No change☐ Decreased by ________

210

8.3 - CTE honor courses offering a more rigorous curriculum for advanced learners

Number of CTE honor courses that are inherently honors or locally approved portfolios aligned to the NC Honors Implementation Guide

CTE Honors Portfolios PowerSchool Inherently Honors Courses

☐ Increased by ________☐ No change☐ Decreased by ________

210

Standard 8 Total (max. 6 points)Overall Total (max. 80 points)


Standard # Indicator # Strategies for Improvement Target Date

1.

2.

3.

1.

2.

3.

1.

2.

3.

1.

2.

3.

1.

2.

3.

1.

2.

3.

CTE Program Review Opportunities for Improvement

Documents

North Carolina CTE Program Eval Instrumentschd.ws/hosted_files/2017springmeetingagenda/88/North... · Web viewProgram Evaluation Instrument Career and Technical Education Guide for