Paths to QUALIITY Using Evaluation Data for Program and Policy Decision Making: Indiana’s Approach...

Paths to QUALIITY

Using Evaluation Data for Program and Policy Decision Making:  Indiana’s Approach to

Implementing a Quality Rating and Improvement System

Presenters

★ Melanie Brizzi

Child Care Administrator, Indiana Bureau of Child Care,

Division of Family Resources★ James Elicker, Ph.D.

Associate Professor, Director, Paths to QUALITY Evaluation, Department of Human Development & Family Studies, Purdue University

★ Pattie Ryan

Deputy Director, Indiana Association for Child Care Resource & Referral

Objectives

★ Paths to QUALITY™ 101

★ Examination of Evaluation Results★ Parents; Providers; Children

★ Data★ Use of results for implementation on state and local level to

strengthen QRIS★ Share future considerations and actions

★ Questions and Answers

History of Paths to QUALITY

★ Homegrown Model★Two local CCR&R agencies

★2000: Early Childhood Alliance (NE Indiana)★2005: 4 Cs Southern Indiana (SW Indiana)

★ Statewide Planning Process

★ Rolled Out Statewide in Four Waves★January, 2008 Wave 1★January, 2009 Last Wave, 4

A True Collaboration

★ Indiana Family and Social Services Administration (FSSA), Bureau of Child Care (BCC)

★ Indiana Association for Child Care Resource and Referral (IACCRR)

★ Indiana Association for the Education of Young Children (IAEYC)

★ The Consultants Consortium (TCC)★ Purdue University Human Development and Family

Studies★ Head Start Collaboration Office★ The Indiana Department of Education

Additional Partners

★ United Way ★ Success by Six★ Ball Brothers Foundation★ Lilly Endowment★ Child and Adult Care Food Program★ Indiana Department of Environmental Management★ Lowes ★ Hagerman Construction★ Halstead Architects★ Working Dog Marketing and Public Relations

Positive child cognitive, language, and social emotional outcomes.

Low quality child care programs either improve their quality of care or close.

More children receive high quality developmentally appropriate child care.

Parents select high quality over low quality child care.

Good and high quality programs follow Paths to QUALITY

requirements, maintaining quality child care.

Parents’ awareness and utilization of Paths to QUALITY rating system.

Programs continue to join Paths to QUALITY and utilize technical assistance support provided by Paths to QUALITY rating system.

Paths to QUALITY quality rating and improvement system implementation.

Ultimateoutcomes

Longer-termoutcomes

Short term/ intermediate outcomes

Inputs

Paths to QUALITY Primary Goals

1. Provide an easily recognizable and understandable tool to help families with their child care choice

CCR&Rs support families for the purpose of increasing access to high quality child care and providing information about the child care

options available to them.

2. Expand the quantity and access to high quality child careCCR&Rs support providers for the purpose of increasing access to high quality child care for all Indiana’s children.

3. Provide a system of resources and supports to help providers increase quality

CCR&Rs support providers to enhance the quality of child care they are offering, through the provision of professional development, technical assistance, and access to resources.

Committee Structure:

Paths to QUALITY Steering Committee

Provider Resources Paths to QUALITY

Marketing and Outreach Policy and Procedure Data Evaluation Software Steering

Standards Work Group

The Process

Understanding the Levels

Validity Report

★ Preliminary assessment of Paths to QUALITY standards validity by Purdue University

★ Reviewed evaluation results of regional Paths to QUALITY programs

★ Are the quality indicators in the proposed statewide Paths to QUALITY standards supported by the scientific literature?

★ Conclusion: “If implemented with diligence…the Paths to QUALITY program has the potential to increase child care quality…[and] we can expect Indiana’s young children to benefit..”

Rating System

★ Independent team of raters★ Providers receive annual visits and visits at level

increases★ Raters utilize the same checklist★ Alignment Committee to ensure that everyone measures

the items in a reliable way★ Inter-rater reliability

2128 Participating Programs

★ Licensed Homes★Goal 55% of licensed homes

★Actual Y3 enrollment 56% or 1568 homes

★ Licensed Child Care Centers★Goal 85% of licensed centers

★Actual Y3 enrollment 86% or 495 centers

★ Registered Ministries that have met Voluntary Certification Standards★Goal 5% registered ministries

★Actual Y3 enrollment10% or 65 ministries

75%

13%5%7%

Total # of Providers as of 10/1/2009

Level 1Level 2Level 3Level 4

68%12%

12%

8%

Total # of Providers as of 10/1/2010

Level 1Level 2Level 3Level 4

52%

24%

14% 10%

Total # of Providers as of 10/1/2011

Level 1Level 2Level 3Level 4

A Closer Look at Paths to QUALITY Evaluation ResearchOverview of Results

Evaluation Questions

★ Do the Paths to QUALITY levels standards and ratings discriminate real differences in child care quality?

★ What are the experiences of child care providers in Paths to QUALITY?

★ How do parents perceive Paths to QUALITY?

★ How are children doing in Paths to QUALITY?

Does Paths to QUALITY actually improve child care quality?

★ We found strong evidence that the Paths to QUALITY rating system distinguishes levels of quality.★Level 4 providers are clearly higher quality than Level 1

providers.

★ There is room for improvement in quality at Paths to QUALITY Level 4.

★ Quality we observed at each Paths to QUALITY Level is quite variable.

ERS Results

Quality Indicator Study:

★ Areas of Quality Strength--★Provisions, supervision, communications with parents and

staff, professional growth★Supervision of play, discipline★Daily schedule, free play, group play

★ Areas for Potential Quality Improvement--★Sanitation in infant/toddler care, preschool★Active physical play★Learning curriculum and interactions: blocks, nature,

science activities, support for language and thinking

Improving Quality: Recommendations

★ Continue to refine Paths to QUALITY rating system: ★Reassess rating instruments to make sure the criteria are

rigorous★Regularly monitor the precision and consistency of Paths

to QUALITY ratings statewide

★ Consider Paths to QUALITY standards revisions/refinements that would elevate overall quality

Statewide Response

★ Celebrating Successes

★ System approach to improving quality★Professional Development★Technical Assistance★Licensing ★Stakeholder Committees

Local Response

★ Technical Assistance★More than 30,000 hours since Paths to QUALITY rolled out

★ Fidelity to the process

★ Case Management System

★ Support for TA Specialists and their supervisors

Why do Providers Enroll?

★ To improve quality, gain recognition, increase business

★ Mentoring and public recognition were seen as most important benefits of participation in Paths to QUALITY

Incentives and Awards

Levels 1-3 Level 4

★ 2008 119,600 58,500★ 2009 237,150 104,400★ 2010 249,400 159,200★ 2011 270,000 127,000

★ Total 876,150 449,100

★ Grand Total $1,325,250

State and local response

★ Grassroots marketing

★ Providing local and state level recognition

★ Addressing “myths” in a systematic way

Yard Signs

Banners

Specifically….

★ Local newspaper ads and other press recognizing participating programs and level advancement

★ Low cost recognition: ribbons for conferences; CCR&R newsletters; social media

★ Marketing toolkit

NACCRRAware

★ Highest rated appear first on referrals

★ Explanation of each rating

★ Change in profile language

★ Coming soon! Child Care Online profiles for programs

Provider Challenges toPaths to QUALITY Advancement★ Time

★ Training

★ Funding

★ Documentation

★ Time Between Rating Visits

State and Local Response

★ On Line Trainings★ Webinars★ Professional Development Network★ Blended T/TA Model for training purposes★ Training Summary Form- move to wireless webforms

Parent Awareness

★ General awareness of Paths to QUALITY among parents is about 14% statewide★35% Early Childhood Alliance★45% 4 C Southern Indiana

★ Child care providers are parents’ most common source of information about Paths to QUALITY

★ Up to 71% of families said Paths to QUALITY would influence their future child care decisions

★ More than 50% are willing to pay more for higher-rated providers

CCR&R Findings

Families surveyed after receiving consumer education and referral services

★ 90% of families state they know more about how to use Paths to QUALITY in their child care search

★ 85% of families view Paths to QUALITY rating as an important or extremely important factor when searching for child care

★ 72% of families report they have or intend to look for a Paths to QUALITY program

★ 55% of families report selecting a Paths to QUALITY provider

Branding