Understanding the NRS Rosemary Matt NYS Director of Accountability

Understanding the NRSUnderstanding the NRS

Rosemary Matt

NYS Director of Accountability

NRS?NRS?

National

Reporting

System

From where did it come?From where did it come?

1998 Federal LegislationWorkforce Investment ActTitle II = Adult EducationImplemented in 1999NYS began in 2001First NRS Report in 2003

AccountabilityAccountability

All states must report on common reporting measures

All states must use US DOE approved standardized (normed) assessments– GAIN has not yet been approved for NYS

All states must report performance annually

New York State DecisionNew York State Decision

All funding sources, in addition to WIA, both state and Federal will be reported under the same guidelines as the NRS

EPE – Employment Preparation Education

WEP – Welfare Education ProgramALE – Adult Literacy Education

Educational GainEducational Gain

Students must meet THRESHOLD CRITERIA

– Have a valid pre assessment– 12 Contact Hours

Measuring Educational GainMeasuring Educational Gain

Students are assessed in BOTH Reading and Math

Their LOWER skill level determines their placement in NRS

Educational Gain is measured and counted ONLY in the LOWER skill level

NRS Level National Reporting System (NRS)Levels

Grade Level Equivalents

1 Beginning ABELiteracy

0 – 1.9

2 Beginning BasicEducation

2 – 3.9

3 Low Intermediate Basic Education 4 – 5.9

4 High Intermediate Basic Education 6 – 8.9

5 Low Adult Secondary Education 9 – 10.9

6 High Adult Secondary Education

11 – 12.9

Program Evaluation ReportProgram Evaluation Report

Enrollment includes students meeting the NRS threshold criteria:

•12 Hours of contact time + Key Demographics

•Valid Pre Test

NYS Policy under NRSNYS Policy under NRS All 5 key demographics must be complete

– Name, birth date, ethnicity, gender, employment status

Student has a VALID pre test Must match Instructional Code

– TABE = BE, GE, or Ma– BEST Plus/BEST Literacy = ES (Spanish GED)

Student is enrolled in a BE, GE, Ma or ES Instructional Activity in current fiscal year

Student must have at minimum of 12 contact hours (this includes any INTAKE hours)

What is a valid pre-test?What is a valid pre-test?

Must be within the scoring range on the NYS Validity Tables

Must match the Instructional Offering Code– ESL Code = BEST + or BEST + and BEST Literacy– BE or GE Code = TABE

Must have a valid date range. A pre test in FY11/12 must be dated between January 1, 2011 and June 30, 2012

NYS Validity Tables (sample)NYS Validity Tables (sample)

Invalid TestsInvalid Tests

Educational GainEducational Gain

Number Making Gain includes students who have a valid pre test and valid post test showing improvement of at least one Educational Functioning Level.

What’s a valid post test? What’s a valid post test?

Must be within the scoring range on the NYS Validity Table

If using the TABE, must be the opposite form (Pre on a 9, then Post on a 10)

Must always move levels contiguously (Pre on one level, then post on the same level or one higher, never backward)

Must have a valid date range. A post test in FY11/12 must be dated between July 1, 2011 and June 30, 2012

Educational GainEducational Gain

Post Test Rate represents the number of students who have a valid post test score divided by the number of students enrolled at each Educational Functioning Level. The Post Test must meet all the requirements of a valid test but it does not necessarily need to show gain to be counted in the post test rate.

65%

Educational GainEducational Gain

Average Contact Hours is a simple average of all cumulative contact hours in a fiscal year divided by the number of students enrolled at each Educational Functioning Level.

Primary Goals:Primary Goals:

Primary Goals: 1. Improve Basic Literacy Skills 2. Improve English Literacy Skills 3. Obtain a job 4. Retain current job 5. Improve current job 6. Earn GED Certificate 7. Earn a Secondary School Diploma 8 Enter Post Secondary Education 9. Enter Training

Secondary Goals:Secondary Goals:

10. Obtain Citizenship Skills 11. Reduce Public Assistance 12. Get involved in Community Activities 13. Get involved in Children’s Education 14. Get involved in Children’s Literacy

Activities 15. Vote or Register to vote

Follow Up OutcomesFollow Up Outcomes

Number of Exiters with Goal:

For Enter Employment this column will include those students having the goal of “Obtain Employment” and exited in Quarters 2, 3, and 4 of FY10/11 AND students exiting in Quarter 1 of FY11/12.

For Retain Employment this column will include those students having the goal of “Retain Employment” and exited in Quarter 4 of FY09/10 AND students exiting in Quarters 1, 2, and 3 of FY10/11.

For Enter Postsecondary Education or Training AND Obtain GED, this column includes students with these goals who have exited in any Quarter of FY11/12.

Follow Up OutcomesFollow Up Outcomes

Percentage Achieving Outcome:

This column represents those students who have responded in a positive way to the required program Survey

Follow Up OutcomesFollow Up Outcomes

NYS FY11/12 Averages:

This is the Benchmark for Follow Up Outcomes in NYS. To arrive at this number, all students achieving the outcome from any of the 4 Goal areas are combined and divided by the sum of all students enrolled with goals who have exited.

Goal Setting Goal Setting

Goal Setting:

Special attention should be given to the determination of a students Employment Status

•Unemployed and Available for Work

•Unemployed and Not Available for Work

Goal SettingGoal Setting

Goal Setting:

NYS Target must be met to earn any points in the category

Goal Setting Goal Setting

Goal Setting:

NYS Target must be met to earn any points in the category

Survey Response RateSurvey Response Rate

Survey Response Rate:

Includes students from:

•Enter Employment

•Retain Employment

•GED Goal

•Postsecondary Education

Remember:

Even a Negative Response is counted as a Response!

R. Matt/NYS Director of Accountability

27

NRS ChangesNRS Changes

R. Matt/NYS Director of Accountability

28

Change Effective Change Effective July 1, 2012July 1, 2012

OVAE has accepted major change to NRS OVAE has accepted major change to NRS GuidelinesGuidelines

All proposed changes begin July 1, 2012All proposed changes begin July 1, 2012

ASISTS will make changes in data system ASISTS will make changes in data system to begin July 1, 2012to begin July 1, 2012

R. Matt/NYS Director of Accountability

29

11stst Major Change Major Change

R. Matt/NYS Director of Accountability

30Let’s compareLet’s compare

Current System:Students are assessed at IntakeDetermination is made:

Is student employed __yes __noIs student available for work?__yes __no

If YES, then assign the goal of “Obtain Employment” (70% Target)If NO – then student status is coded as “Not in the workforce”

Changes to System:Students are assessed at IntakeDetermination is made:

Is student employed __yes __noIs student available for work?__yes __no

If YES, then ASISTS will automatically assign the goal of “Obtain a Job” to all students in this cohortIf NO – then student status is coded as “Not in

the workforce”

R. Matt/NYS Director of Accountability

31

Current System:Students are assessed at IntakeDetermination is made:

Is student employed __yes __no

If YES – is student enrolled in literacy to Retain his/her employment?

If YES – code student with goal of “Retain Employment”

Changes to System:Students are assessed at IntakeDetermination is made:

Is student employed __yes __no

If YES, then ASISTS will automatically assign the goal of “Retain Employment” to all students in this cohort. Program must conduct Follow Up Survey on each student 3 Quarters after student exits the program.

Let’s compareLet’s compare

R. Matt/NYS Director of Accountability

32

22ndnd Major Change Major Change

R. Matt/NYS Director of Accountability

33

Current System:Students are assessed at IntakeDetermination is made:

Does student have the goal of attaining a High School Equivalency Diploma __yes __no

If YES, then assign the goal of “Obtain GED”

Changes to System:Students are assessed at IntakeDetermination is made:

Programs must report the total number of enrolled students who take all the GED tests during the year and the number who pass them to calculate a pass rate.

All GED test data will now be recorded in ASISTS and a pass rate will be calculated for each program.

Let’s compareLet’s compare

R. Matt/NYS Director of Accountability

34

R. Matt/NYS Director of Accountability

35

33rdrd Major Change Major Change

R. Matt/NYS Director of Accountability

36

Current System:Students are assessed at IntakeDetermination is made:

Does student have the goal of attaining a High School Equivalency Diploma __yes __no

If YES, then assign the goal of “Obtain GED” Second Question – Does the student also have the goal of entering Postsecondary Education or Training? If YES, assign goal (50% target)

Changes to System:Students are assessed at IntakeDetermination is made:All learners who have a GED, a secondary credential, or are enrolled in a class specifically designed for transitioning to postsecondary education (e.g., bridge program, college readiness), as the cohort for which “Entry into Postsecondary Education” must be tracked. Students included in the cohort are those who passed the GED tests or earned a secondary credential while enrolled in adult education, enter with these credentials, or are enrolled in a class specifically designed for transitioning to postsecondary education.

Let’s compareLet’s compare

R. Matt/NYS Director of Accountability

37

Additional ChangeAdditional Change

Entry into Postsecondary Education or Training can occur any time from the time of exit until the end of the following program

year.

R. Matt/NYS Director of Accountability

38

2 Minor Changes2 Minor Changes

Students’ Highest Educational Level or

Degree Attained

&

Teacher Experience and Certification

.

Contact Information:Contact Information:

Rosemary.matt@cayuga-cc.edu