Standards and Data and Outcomes … Oh MY! Moving from Perception to Reality with Data and...

Standards and Data and Outcomes … Oh MY!

Moving from Perception to Reality with Data and Professional Standards

2014 AER Conference

Presenters

Kay Ratzlaff Florida TVI Instructional

Materials Center QPVI Facilitator

Nancy Toelle Texas TVI Texas School for the

Blind QPVI Developer

July 30 – August 3 AER 2014

Discussion for Today Data

Strategies for data collection Managing data Using data for decision making

Professional Standards of Practice

Outcomes

Yes, we have to pay attention to the “man” behind the curtain!

“Gee Toto, I’ve a feeling we’re not in Kansas anymore.”

July 30 – August 3 AER 2014

Goals Identify strategies for collecting and

managing data to enhance program development and instruction

Establish professional development strategies that implement evidence-based practices and encourage program and individual professional growth

Select standards-based tools for conducting essential assessments and communicating results.

Quality Indicators

• Timely Records• Services Meeting the Standard • Services Based on Assessed Needs• Outcomes Measured

Which Characters Do You See in Your Staff?

Dorothy - dreamer who needs to be rescued

Tin Man – thinks with their heart Scarecrow – thinks only with their brain Lion – scared until absolutely necessary Glinda – pops in to make it “all better” Wizard – good intentions, but faulty

methodsJuly 30 – August 3 AER 2014

And yes … Elphaba, the witch, that makes life

miserable for everyone!

July 30 – August 3 AER 2014

How do you move from all these personalities to …

A team working together to make it to the Emerald City?

July 30 – August 3 AER 2014

QPVI in Florida

FIMC-VI supporting and facilitating delivery to districts

Voluntary participation of districts Administrator commitment Self-study model Building Local Capacity model Ongoing support

QPVI in Florida Aligns with increase in accountability Provides data to support programs and

services Provides training for administrators Addressing problems in programs and

practices based on data not perception Included in the Florida Department of

Education Strategic Plan

Data

1. Timely (Compliance)a. Eye medical report

b. Assessments of functional vision and learning/literacy media

c. O&M screenings and evaluations

d. ECC screenings and evaluations

e. IEP and Reevaluation dates

Data2. Student Information

a. Eye condition(s)

b. Acuities and peripheral fields

c. Time in general education

d. Primary and secondary conditions

e. Other services (OT, PT, etc.)

f. Primary and secondary learning media

g. Font sizes and working distance

h. Instructional reading levels and WPM

Data

3. Program a. Service – direct or consult

b. Minutes of direct service

c. Minutes of consult or support

d. Paraeducators or braillist

e. Clinical low vision evaluations

f. FIMC-VI registrations

Master List Example

DATA FROM MASTER LIST

Example of Data-Driven Decision Making!

Direct Versus Consult Services

55

38

1

21

DirectConsult504Blanks

Direct = 55Consult = 38504 = 1Blank = 21

Comparison Data:Total Students 2009 - 2012

From 61 to 83 Direct Students = 31% to 33%From 136 to 169 Collaborative Con. Students = 69% to 67%

Florida Sample Data – From QPVI Master List of 473 Students

Approximately 87% of students have more than one eye condition

45% of students have Visual Impairment as their primary disability

55% have another disability as primary 44% are labeled Intellectually Disabled (IND) 10% are labeled Orthopedically Impaired 8% are labeled Developmentally Delayed 7% are labeled Deaf Hard of Hearing or Dual Sensory

Impaired; or Hospital Homebound, or Other Health Impaired

6% are labeled Specific Learning Disability (SLD) 4% are labeled Autistic

Common Visual Impairments Nystagmus (26%) Myopia (15%) Optic Nerve Atrophy

or Hypoplasia (13%) Cortical Visual

Impairment (10%) Retinopathy of

Prematurity (8%) Ambylopia (8%)

High Myopia (8%) Glaucoma (6%) Albinism – Ocular

and Oculocutaneous (5%)

Syndromes (5%) Colobomas (3%) Cataracts (3%) Aphakia (2%)

6th Grade FCAT Data on Reading

0

10

20

30

40

50

60

70

80

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Grade 6 VI

Grade 6 All

Grade 6 ESE

Percent of students scoring a 3 or better.

Service Delivery Compared by Florida District

 QPVI Self-Study:

The results of each Key Component are applied to the Master List

Master List Eligibility Unique Needs Roles & Responsibilities Type/Amount Staff/Caseloads

1. Baseline data (snapshot) is archived

2. Data is analyzed by VI staff to determine compliance problems and establish priorities

3. Action plan is designed with names, assignments, due dates, completion targets

Process after Data Collection

Sustaining the Data1. Establish a process for program data

collection (shared drives, paper version to clerical person, etc.)

2. Update the student data after each IEP meeting

TVI and O&M

Use the data for: Eligibility decisions Instructional goals Type, time, and intensity of

services Progress monitoring

Program Supervisor

Use the data for Caseload management Tracking results on action

plans Monitoring student

outcomes Justification for supplies,

equipment, and personnel Consolidated data

Noting Presence/Timeliness of VI Program Data

Determine most critical data points to reflect data-driven decision making

Identify data points that are critical to reflect quality programming (eye reports, FVA, LMA, etc.)

Develop Master List: Excel/Access or Numbers

Complete, analyze, set priorities, sustain

District Baseline & Year One Comparison – Target 100%

Dist 1 Dist 2 Dist 3 Dist 4 Dist 5Baseline 48% 64% 78% 65% 62%Current 87% 92% 100% 71% 85%

0%

20%

40%

60%

80%

100%

120%

Timely Functional Vision Assessments

Key Components – Addresssing Issues of “Quality”

1. Developing a master list of students database

2. Eligibility for VI services

3. Unique VI student needs

4. Roles and responsibilities

5. Type and amount of VI service

6. Staff and caseloads

7. Self-study report

Let’s analyze this master list

What’s wrong with this picture???

1. Analyze and aggregate the dataa. What are the good data points?

b. What are the bad data points?

2. How would you use that data to drive your decisions or change your practices?

Why is Assessment Data Important???

9/18/201233

Professional Standards for TVIs and O&Ms

Assessments that align with the EA Rubric

Instruction that implements the Expanded Core Curriculum

Data-driven decision making Needs of students determine service Use of published and peer-reviewed

materials and curricula

Professional Practices for TVIs and O&M

Use of research-based instructional practices Delphi studies on instructional time for

literacy instruction Braille readers Readers with low vision

ABC Braille Study for instructional methods

Professional Practices for TVIs and O&Ms

RIOT model for reevaluations (Record Review, Interview, Observation, Test)

Assessments of functional vision, learning media, and ECC areas are conducted with fidelity

Professional Practices for TVIs

Peer-review of all reports Teaching ECC areas Collecting data on instructional

outcomes AS EVIDENCE OF PROGRESS

Using data to drive instruction and goals

Strategies to Make it Happen

Intensive time for training Most school districts take 3 to 4 years to

implement the changes Follow-up on consistent basis “Inspect what you expect” – hold people

accountable Regular Communication

Wikis, shared drives, small group meetings, mentors, etc.

To Make it Work … 1. Data collection process2. Commitment of administrators3. Established professional standards of

performance4. Commitment of administrators 5. Strategies for Communication:

Wikis, shared drives, small group meetings, mentors, etc.

6. Did we say … Commitment of administrators to enforce standards!

Accountability Is Here To Stay Make It Count Use professional

resources - no more flying by the seat of our pants!

Engage in professional development - conferences, webinars, internet

Know your stuff Be a professional

Do It Right Implement the strategies

that work Take responsibility for

outcomes of students (no more blaming them!)

Be “in the room and at the table” to advocate for students and clients

Data… Professional Standards … Quality … Instruction … Assessment ... Outcomes

Establish goals for quality and train to meet those goals

Decide on your data points and processes. COLLECT AND ANALYZE DATA TO TRACK PROGRAM QUALITY

Clearly establish your professional standards

Peer reviews, “data days,” changes in practices, evaluations

Process is dynamic and not static, adjust as needed

Action Plan

DataData

StandardsStandards

QualityQuality

ImplementImplement

ReviewReview

You are the professional in the room!

Resources Assuring Quality Literacy Instruction for Students Who are Blind

 by: Alan J. Koenig • Texas Tech University, and M. Cay Holbrook • The University of British Columbia; From: Koenig, A.J., & Holbrook, M.C. (2000). Ensuring high-quality instruction for students in braille literacy programs. Journal of Visual Impairment and Blindness, 94, 677-694. Available for download: www.nfb.org/images/nfb/documents/.../delphibraillestudentsholbrook.doc

Literacy for Students with Low Vision: A Framework for Delivery of Instruction, by  Anne L. Corn • Vanderbilt University, and Alan J. Koenig • Texas Tech University: From: Journal of Visual Impairment and Blindness, May 2002, 96, . Available for download: http://www.afb.org/jvib/JVIB960503.asp

Quality Programs for Students with Visual Impairments, Nancy Toelle, developer: www.qpvi.com

Essential Assessments Rubric, www.earubric.com

9/11/2007 FIMC-VI

Facilitators for Today’s Session

Nancy Toelle

QPVI Developer / Texas School for the Blind and Visually Impaired

nancytoelle@gpvi.com & www.QPVI.com Kay Ratzlaff

Coordinator of Instructional Resources

Florida Instructional Materials Center for the Visually Impaired

kratzlaff@fimcvi.org & www.FIMCVI.org

July 30 – August 3 AER 2014