11 Don’t Be Afraid of the New Tech Prep Accountability! ! NACTEI CONFERENCE Boise, Idaho May 21 & 22, 2008

11

Don’t Be Afraid of the New Tech Prep Accountability! !

NACTEI CONFERENCE

Boise, Idaho

May 21 & 22, 2008

22

PRESENTER

Mr. Dennis Fiscus

NATPL President

Arizona

State Tech Prep Director

33

OUTLINE

How Can You Make Tech Prep Data Collection and Reporting Work For You

Tech Prep and Perkins IV How to Make Data Collection Work How to Approach the Data Collection of Tech

Prep Indicators of Performance Questions/Discussion

What is a accountability?What is Tech Prep accountability?Why do we need accountability?

How do we collect data?How do WE keep Tech Prep going?

DATA!

55

Sec. 113. Accountability

Perkins IV requires data to be collected for career and technical education

Core indicators of performance at secondary and postsecondary level identified

Levels of performance identified for each core indictor, expressed in percentage and numerical form

66

Sec. 113. Accountability

Tech Prep is included in all section 113 language

Tech Prep is always a subset of the Perkins Performance Measures

Accountability is nothing new. . . States are already collecting data

Perkins IV

Core Indicators of Performance (SEC. 113. Accountability)

Collect and report data on a state and national level Consistent data collected for reporting purposes Reports program and student outcomes Identifies “indicators of performance” for accountability

purposes Regardless if states combine “funds”, important to

collect and report Tech Prep data

Sec. 113. Accountability-SECONDARY

1S1: ATTAINMENT OF ACADEMIC SKILLS - READING/LANGUAGE ARTS

1S2: ATTAINMENT OF ACADEMIC SKILLS – MATHEMATICS

2S1: ATTAINMENT OF TECHNICAL SKILLS 3S1: SECONDARY SCHOOL COMPLETION 4S1: STUDENT GRADUATION RATES 5S1: SECONDARY PLACEMENT 6S1: NONTRADITIONAL PARTICIPATION 6S2: NONTRADITIONAL COMPLETION

Sec. 113. Accountability POSTSECONDARY

1P1: ATTAINMENT OF TECHNICAL SKILLS 2P1: ATTAINMENT OF CREDENTIAL,

CERTIFICATE, OR DIPLOMA 3P1: STUDENT RETENTION OR TRANSFER 4P1: STUDENT PLACEMENT 5P1: NONTRADITIONAL PARTICIPATION 5P2: NONTRADITIONAL COMPLETION

Student Accountability Forms for the Section 113 Core Indicators of Performance (Title I) SECONDARY LEVEL

1S1: ATTAINMENT OF ACADEMIC SKILLS - READING/LANGUAGE ARTS

STATEPROGRAM YEAR

Amended Performance Data Date of Filing Amended Data: _______________________

1 GRAND TOTALPre-populated from

the FAUPL

23 Male 4 Female

5

6 American Indian or Alaskan Native7 Asian or Pacific Islander8 Black (not Hispanic)9 Hispanic

10 White11 Unknown

1213 American Indian or Alaska Native14 Asian15 Black or African American16 Hispanic/Latino17 Native Hawaii or Other Pacific Islander18 White 19 Two or More Races2021 Individuals With Disabilities (IDEA)22 Economically Disadvantaged23 Single Parents24 Displaced Homemakers25 Limited English Proficient26 Migrant Status27 Nontraditional Enrollees28 Tech Prep

RACE/ETHNICITY* (1977 Standards)

GENDER

Line Population

*See "Definition of Terms" for guidance with reporting the Race and Ethnicity Categories.

Number of Students in the Denominator

Number of Students in the

Numerator

State Adjusted Level of

Performance

Actual Level of Performance

Adjusted vs. Actual Level of Performance

Met 90% of Adjusted Level of

Performance (Y,N)

RACE/ETHNICITY* (1997 Revised Standards)

SPECIAL POPULATIONS AND OTHER STUDENT CATEGORIES

Student Accountability Forms for the Section 113 Core Indicators of Performance (Title I) SECONDARY LEVEL

1S2: ATTAINMENT OF ACADEMIC SKILLS - MATHEMATICS

STATEPROGRAM YEAR


1 GRAND TOTALPre-populated from

the FAUPL

2 GENDER3 Male 4 Female


10 White11 Unknown

1213 American Indian or Alaska Native14 Asian or Pacific Islander15 Black or African American16 Hispanic/Latino17 Native Hawaiian or Other Pacific Islander18 White 19 Two or More Races2021 Individuals With Disabilities (IDEA)22 Economically Disadvantaged23 Single Parents24 Displaced Homemakers25 Limited English Proficient26 Migrant Status27 Nontraditional Enrollees28 Tech Prep



SPECIAL POPULATIONS AND OTHER STUDENT CATEGORIES


Line PopulationNumber of

Students in the Numerator


State Adjusted Level of

Performance

Actual Level of Performance

Adjusted vs. Actual Level of Performance

Met 90% of Adjusted Level of

Performance (Y,N)

Student Enrollment Forms Enrollment of CTE Participants

STATEPROGRAM YEAR


Number of Adult Students

Number of Secondary Tech Prep Students

1 GRAND TOTAL 23 Male 4 Female


10 White

11 Unknown

1213 American Indian or Alaska Native14 Asian15 Black or African American16 Hispanic/Latino17 Native Hawaiian or Other Pacific Islander18 White19 Two or More Races

2021 Individuals With Disabilities22 Economically Disadvantaged23 Single Parents24 Displaced Homemakers25 Limited English Proficient26 Migrant Status27 Nontraditional Enrollees


Line PopulationNumber of

Postsecondary Students

Number of Secondary Students

Number of Postsecondary

Tech Prep Students

SPECIAL POPULATION AND OTHER STUDENT CATEGORIES


GENDER


Student Accountability Forms for the Section 203 Indicators of Performance (Title II)

SECONDARY LEVEL

Amended Performance Data Date of Filing Amended Data: __________________

1 1STP1 Enroll in postsecondary education

2 1STP2 Enroll in postsecondary in the same field or major

3 1STP3 Complete a State or industry-recognized certification or licensure

4 1STP4 Complete courses that award postsecondary credit

5 1STP5 Enroll in remedial mathematics, writing, or reading courses

PROGRAM YEARSTATE

Indicator Number

Performance IndicatorPercent of Students



NumeratorLine

Student Accountability Forms for the Section 203 Indicators of Performance (Title II) POSTSECONDARY LEVEL


1 1PTP1 Enployment after graduation

2 1PTP2 Complete a State or industry-recognized certificate or licensure

3 1PTP3 Complete a 2-year degree or certificate

4 1PTP4 Complete a baccalaureate degree program

STATEPROGRAM YEAR

LinePercent of Students

Indicator Number

Performance IndicatorsNumber of



1515

Why the need to collect and report Tech Prep data: To provide relevant information to

stakeholders To respond to the ongoing concerns and

issues raised by stakeholders To respond to legislation: Sec 203. Indicators

of Performance and Accountability To use data to identify what is working and

what needs improvement To sustain funding

16

In order to make your Data System work for you, you will need:1) To want to do this,2) Develop workable secondary and

postsecondary systems to report Tech Prep data,

3) Budget dollars to support data collection and reporting of results,

4) Buy-in from all stakeholders,5) Active consortia to deliver Tech Prep, 6) Investment in technology to simplify the

collection and analysis of data,

7) A method for collecting and reporting data needed to adequately report to OVAE for the Consolidated Annual Report (CAR),

8) Coordination with agencies that have employment and earnings data (e.g. FEDES),

9) To share results of program improvement and success efforts with all stakeholders,

10) To conduct a small ,achievable pilot project if no data collection system currently exists in your state.

17

In order to make your Data System work for you, you will need:

1818

Overview State Data Collection Systems – Arizona

SECONDARY

Integrated with existing Career and Technical Education student reporting system

Tech Prep data is a subset of all CTE reports and Performance Measures

Long time established Tech Prep Consortia State Tech Prep Student definition makes it work Signed Articulation agreements mandatory

1919


POSTSECONDARY

Uses secondary data to develop Postsecondary Tech Prep student list

Tech Prep Consortia Directors an integral part of data system

Postsecondary Institutional Research people a key Tech Prep developed a statewide postsecondary,

web-based system for reporting AZ Department of Education did all of the training

2020


STEPS

1. Tech Prep Directors submit to State Articulation reports to determine TP programs/students

2. Secondary LEA’s report to AZ Department of Education

3. Programs are tagged as Tech Prep or Not Tech Prep

4. Students fall into programs that are tagged

5. Completers come from programs that are tagged

2121


STEPS

6. State completer list available to Tech Prep Directors

7. Tech Prep directors work with Institutional Research people at Community College to find Tech Prep completers

8. Students that are found are then tagged as Postsecondary Tech Prep students

9. Postsecondary Tech Prep students then become a subset of the postsecondary reporting system

10. Tech Prep Consortia Directors watch over the system

2222

How to Approach the Data Collection of Tech Prep Indicators of Performance

Section 205. Report “report on the effectiveness of tech prep programs”

Tech Prep Indicators Definitions (Numerators and Denominators) Secondary Tech Prep Student: a secondary

education student who has enrolled in 2 courses in the secondary education component of a tech prep program.

2323

How to Approach the Data Collection of Tech Prep Indicators of Performance

Tech Prep Indicators Definitions (Numerators and Denominators) Postsecondary Tech Prep Student: a student

who (A) has completed the secondary education component of a tech prep programs; and (B) has enrolled in the postsecondary education component of a tech prep program at an institution of higher education described in clause (i) or (ii) of section 203 (a)(1)(B).

Perkins IV - Indicators of Performance (SEC. 203. Tech Prep Program, Indicators of Performance and Accountability)

(A) The number of secondary education tech prep students and postsecondary education tech prep student served.

(B) The number and percent of secondary education tech prep students enrolled in the tech prep program who

(i) enroll in postsecondary education;(ii) enroll in postsecondary education in the same field

or major as the secondary education tech prep students were enrolled at the secondary level;


(iii) complete a State or industry-recognized certification or licensure;

(iv) successfully complete, as a secondary school student, courses that award postsecondary credit at the secondary level; and

(v) enroll in remedial mathematics, writing, or reading courses upon entering postsecondary education.


(c) The number and percent of postsecondary education tech

prep students who(i) are placed in a related field of employment not later than 12 months after graduation from the tech prep program; (ii) complete a State or industry-recognized certification or

licensure;(iii) complete a 2-year degree or certificate program within the normal time for completion of such program; and(iv) complete a baccalaureate degree program within the normal time for completion of such program.

Student Accountability Forms for the Section 203 Indicators of Performance (Title II)

SECONDARY LEVEL


1 1STP1 Enroll in postsecondary education

2 1STP2 Enroll in postsecondary in the same field or major

3 1STP3 Complete a State or industry-recognized certification or licensure

4 1STP4 Complete courses that award postsecondary credit

5 1STP5 Enroll in remedial mathematics, writing, or reading courses

PROGRAM YEARSTATE

Indicator Number

Performance IndicatorPercent of Students



NumeratorLine

Student Accountability Forms for the Section 203 Indicators of Performance (Title II) POSTSECONDARY LEVEL


1 1PTP1 Enployment after graduation

2 1PTP2 Complete a State or industry-recognized certificate or licensure

3 1PTP3 Complete a 2-year degree or certificate

4 1PTP4 Complete a baccalaureate degree program

STATEPROGRAM YEAR

LinePercent of Students

Indicator Number

Performance IndicatorsNumber of



29

(1) The number of secondary education tech prep students and postsecondary education tech prep students served.

Measurement Definition

Number of secondary TP students

Number of postsecondary TP students

Collected & Percent

In Sec. & PS

# of TP Students

3030

(2) The number and percent of secondary education tech prep students enrolled in the tech prep program who- “(i) enroll in postsecondary education.

Numerator

#of secondary TP students who graduated last year and are enrolled during the current year

Denominator

# of secondary TP students who graduated last year

Collected & Percent

In PS

% of secondary TP enrolled in PS

31


Denominator


Must know this number/student first

Already collected as a part of the state CAR report for Technical Attainment-denominator

32


Denominator


Student file utilized could contain just a common identifier such as SSN or Department of Education state student identifier

33


Denominator


File could contain common student identifier fields such as: Student name High school name Gender Graduation date/year CTE Program

completed

34


Denominator


This finds the majority of the students

Some “hands on” delineation or sorting is required

35


Numerator


Run list of secondary tech prep graduates through the community college enrollment database

Utilize National Clearinghouse especially for universities and out of your state enrollees

36


Numerator


Once found-need to place an identifier with the postsecondary student record to show this is a Tech Prep student

(XX) Tech Prep

3737

(3) The number and percent of secondary education tech prep students enrolled in the tech prep program who- “(ii) enroll in postsecondary education in the same field or major as the secondary education tech prep students were enrolled at the secondary level.

Numerator#of secondary TP students

enrolled in the current year in the same major or cluster/pathway as in high school the previous year same graduation

Denominator# of secondary TP

students who graduate last year

Collected & PercentIn PS

% of secondary TP enrolled in same filed in

PS

38


Denominator

# of secondary TP students who graduate last year

Again-This is the same number as from your CAR report-Technical Attainment denominator

Same as the denominator above for #2

39


Numerator

#of secondary TP students enrolled in the current year in the same major or cluster/pathway as in high school the previous year same graduation

This is a subset of the numerator above on #2

The subset is now needing JUST those in same major/cluster

40


Numerator


Your state MUST be identifying what programs the secondary students are in-by CIP and program name

# of students BY program is reported in the CAR on form 3

41


Numerator


Student identifier from secondary should have the students program as part of the identifier field

As before-run student list through the college system to find student

42


METHODS OF DETERMINING MAJOR OR CLUSTER/PATHWAY

Must first define Major and Cluster/Pathway for use in your system

43


WHAT DETERMES A CLUSTER/PATHWAY ? Utilize Classification of Instructional Programs list-

CIP CODE Same first two digits of CIP code=Cluster/Pathways

01.000=Agriculture 51.000=Health

44


WHAT DETERMES A CLUSTER/PATHWAY ? Utilize Classification of Instructional Programs list-

CIP CODE Student has 7 or more credits in a programmatic

area-Business education, welding, computers

45



May need Tech Prep consortia directors to utilize a Student Tech Prep “Registration” form

Form would include students “major area of interest” = MAJOR

46



What if found using clearing house and the student is at a University in OR out of state? If Consortia Director has utilized a Student

Registration form indicating career interest-use it

47



What if found using clearing house and the student is at a University in OR out of state? Assume if no contradicting information is present

and student is continuing to pursue career interest from secondary?

48


Numerator#of secondary TP students

enrolled in the current year in the same major or cluster/pathway as in high school the previous year same graduation

Denominator# of secondary TP

students who graduate last year

We MUST accept the fact that this figure is A subset of a subset It may not be a high

number

4949

(4) The number and percent of secondary education tech prep students enrolled in the tech prep program who- “(iii) complete a State industry-recognized certification or licensure.

Numerator

#of secondary TP students who graduated last year with a State or industry recognized certification, or licensure

Denominator


Collected & Percent

In PS

% of secondary TP with certification or licensure

50


Denominator


Same as #2 above Same as #3 above

51


Numerator


This is a subset of the new Perkins IV Measure 113 (b)(2)(A)(ii) 2S1 Technical Skill Attainment

Subset is part of the NUMERATOR for this 2S1 Perkins Measure

52


Numerator


Your state should already be collecting this data

Your Tech Prep concentrators are just a subset

5353

(5) The number and percent of secondary education tech prep students enrolled in the tech prep program who- “(iv) successfully complete, as a secondary school student, courses that award postsecondary credit at the secondary level.

Numerator

#of secondary TP students graduated last year with postsecondary credits

Denominator


Collected & Percent

In Sec.

% of secondary TP with PS credit

54


Denominator


Same as #2 above Same as #3 above Same as #3 above

55


Numerator


Once again-this is simply a subset of the Denominators above for 2, 3, 4

The subset is defined as the Tech Prep students that GRADUATED secondary WITH postsecondary credits

56


Numerator


Most all states REQUIRE colleges to REPORT: DUAL

ENROLLMENT CREDITS AWARDED TO SECONDARY STUDENTS

57


Numerator


Many high schools and community colleges already have this report for ALL students receiving credits

You need to receive that report and compare (run) your Tech Prep secondary concentrator report against it

58


Numerator


Primary idea behind this Measure is that ALL

Tech Prep articulated programs from secondary to

postsecondary SHOULD have a “for CREDIT” component

5959

(6) The number and percent of secondary education tech prep students enrolled in the tech prep program who- “(v) enroll in remedial mathematics, writing, or reading courses upon entering postsecondary education.

Numerator

#of secondary TP students graduated last year who enrolled in remedial mathematics, writing, or reading courses upon entering postsecondary education

Denominator

# of secondary TP students who graduated last year and enrolled in postsecondary education

Collected & Percent

In PS

% of remedial TP students

60


Denominator

# of secondary TP students who graduated last year and enrolled in postsecondary education

Same number as the NUMERATOR in #2 above

61


Numerator


This is a subset of your Denominator for this measure

Should have electronic access to transcripts at the postsecondary

62


Numerator


Usually courses in academics below “100” level are considered remedial

Sometimes the colleges identify the courses as “Remedial Math”, etc

If it shows up on the Tech Prep students transcripts-then the student is counted in the numerator measure

6363

(7) The number and percent of postsecondary education tech prep students enrolled in the tech prep program who- “(i) are placed in a related field of employment not later than 12 months after graduation from the tech prep program.

Numerator

#of postsecondary TP students placed in a related field no later than 12 months after graduation

Denominator

# of postsecondary TP students who graduated last year

Collected & Percent

In PS

% of grads with certificates, etc.

64


Denominator

# of postsecondary TP students who graduated last year

The Tech Prep Measure Denominator just is a subset of 4P1 Perkins Measure on Student Placement Denominator


Numerator


The only additional element is “related” employment

Define “related” for your circumstances and give to OVAE

If you find them-count them


Numerator


Possible definitions Assume positive-if

placed, it is related Utilized CIP code

with job name Let student tell you it

is related


Numerator


Possible methods of collection Unemployment

Insurance (UI) data from DOL surveys

Student follow-up surveys

FEDES

6868

(8) The number and percent of postsecondary education tech prep students enrolled in the tech prep program who- “(ii) complete a State or industry-recognized certification or licensure.

Numerator

#of postsecondary TP students who leave postsecondary education this year with a State, industry recognized certification, or licensure

Denominator

# of postsecondary TP students who left postsecondary education this year

Collected & Percent

In PS

% of grads with certificates, etc.

69


Denominator


AGAIN This Tech Prep Measure Denominator just is a subset of 4P1 Perkins Measure on Student Placement Denominator

Also same as #7 above

70


Numerator


This is a subset of the Perkins Postsecondary Measure 2P1-Credential, Certificate, or diploma Attainment

The “Perkins” folks should have already figured this one out

Just determine subset “Tech Prep” utilizing Tech Prep identifier

71


Numerator


Most colleges have determined and utilize certifications within their degree programs

Most colleges have this as a part of their report to the State “Governing Board”

72


Numerator


Often the certification and licensures are also reported on the transcripts

Often the college has this as a “termination” point for the students “alternative” to a degree

73


Numerator


Sometimes the student will take an industry test

Utilize a “Tech Prep” agreement with the student to receive the results if your college hasn’t already arranged to receive the results

74


Numerator


Sometimes the student will take an industry test

A “Tech Prep” agreement with the student may include some “incentives” for reporting

A “Tech Prep” agreement with the student may include the results coming to the college

7575

(9) The number and percent of postsecondary education tech prep students enrolled in the tech prep program who- “(iii) complete a 2-year degree or certificate program within the normal time for completion of such a program.

Numerator

#of postsecondary TP students who entered 3 years ago

Denominator


Collected & Percent

In PS

% that graduated within 2 years

76


Denominator


Again-This Tech Prep Measure Denominator just is a subset of 4P1 Perkins Measure on Student Placement Denominator

Also same as #7 & 8 above

77


Numerator


Could be tagged along with the Perkins Measure of Retention/Transfer 3P1

Utilize the Tech Prep identifier to simply compare graduates or certification completers each year to how long ago they entered the institution

78


Numerator


Perhaps we need a discussion on Is a 3 years time

frame “normal” for the community college student to graduate?

That time frame is 5 years in Arizona

Your college data can help determine this

7979

(10) The number and percent of postsecondary education tech prep students enrolled in the tech prep program who- “(iv) complete a baccalaureate degree within the normal time for completion of such a program.

Numerator

#of postsecondary TP students who entered the postsecondary Tech-Prep program 6 years ago and completed a baccalaureate degree program

Denominator


Collected & Percent

In PS

% that graduated within with baccalaureate

80


Denominator


Again-This Tech Prep Measure Denominator just is a subset of 4P1 Perkins Measure on Student Placement Denominator

Also same as #7, 8, 9 above

81


Numerator


Here’s our best thoughts This can be done This will be a very

low number and percentage

We WILL NOT find most of our Tech Prep students

82


Numerator


Here’s our best thoughts Utilize the National

Clearinghouse database

Use your states University system database

Utilize student surveys

8383

Next Steps

Consider using NATPL Quality Indicators as a tool to assess your Tech Prep efforts:

2006 review of existing “quality indicators” Matrix is developed to reflect current and future Tech Prep

practices and accountability Matrix crosswalks with Perkins IV and Core Indicators Five Quality Indicator Components

1) Accountability/Sustainability2) Student Opportunities3) Curriculum4) Articulation5) Professional Development

TECH PREP PROGRAM QUALITY INDICATORSDeveloped by the National Association for Tech Prep Leadership (NATPL)

January 2007, March 2003, February 1999

A Tech Prep Program is comprised of five integral components: (1) Accountability/Sustainability, (2) Student Opportunities, (3) Curriculum, (4) Articulation, and (5) Professional Development. The Perkins Core Indicators include: (A) Student Attainment Academic (S/PS); (B) Student Attainment Career Technical Proficiency (S); (C) Industry Credential (PS); (D) Graduation Rates (S); (E) Placement Postsecondary, Military, Employment (S); (F) Placement Military, Apprenticeship, Employment (PS); and (G) Participation and Completion Nontraditional Fields (S/PS). Note: S = Secondary Indicator, PS = Post-Secondary Indicator. Use coding for 2006 Perkins Core Indicators of Performance Column.

8585

Next Steps:

Meet with State “data people” to discuss Tech Prep Indicators of Performance

Local Tech Prep directors meet with “data people” at community college to discuss next steps

Review your Perkins State plan and be familiar with Section 205 requirement “report on the effectiveness of tech prep programs”

Attend the NATPL Forum and NCPN sessions on Tech Prep data collection

Don’t give up . . . if we all work on this together we will sustain future funding for Tech Prep!

Don’t Be Afraid of the New Perkins Accountability! !

QUESTIONS??

8787

PRESENTER

Mr. Dennis Fiscus NATPL President

Arizona State Tech Prep [email protected]

602.542.5356

Documents

11 Don’t Be Afraid of the New Tech Prep Accountability! ! NACTEI CONFERENCE Boise, Idaho May 21 & 22, 2008