1 QRIS National Learning Network Learning Table Welcome! 1 “Strengthening Inquiry”

1

QRIS National Learning Network Learning Table

Welcome!

1

“Strengthening Inquiry”

Faculty

Garrison KurtzJane Reisman Kasey Langley

Diana Schaack

Instructors

TechnicalSupport

2

Learning Table Overview

3

Expands to...

GoToMeetingParticipant Tools

4

Clicking here...

Ground Rules

• Manage background noise through use of the mute buttons *6 switch between mute and talk

• Listen to understand• Indicate your state and first name when speaking• See each other as resources• Realistic expectations of busy professionals

5

Forum Topics

6

Moving from our current situation to our desired - How do we manage the move from our current state (including whatever “plans or theory” we have articulated), to the new state? How must/should we engage the different players who want/need input and buy-in?

Critical Questioning

What are your initial reactions to the Disarray QRIS Plans?

7

* Goals to increase school readiness and sustainable child care businesses might be at odds with other goals* Since the first four levels are done through self-assessment, they might provide an inaccurate picture of actual quality* The timeline is incredibly fast for launching, carrying them out and evaluating impact* The report to the legislature requires child outcome data, yet there is not clearly any data gathering or evaluation underway* Things sometimes unfold in an unhelpful sequence with odd expectations which can hamper what we really need to do

Case Study – Raising Critical Questions

Assumptions• What are some of the assumptions that are explicit or implicit in

the Disarray QRIS plan? • What is the basis for those assumptions? • Are they valid or should they be revisited?

Accountability• What outcomes are the team designing/implementing

accountable for achieving? • Do the strategies match the outcomes?

8

Levels of PlanningStrategies

9

StandardsProgram/

Practitioner Support

Accountability IncentivesOutreach &

Communications

50,000State of Disarray

QRIS Pilot

Expansion Strategies

Long-term Financial Strategies

Communications Outreach and

Education

30,000

10,000Set Quality Standards

Accountability

State of Disarray30,000 Foot Outcome Map

Standards•Set and promote quality standards

Accountability•Conduct ratings•Gather and manage data

Program/Practitioner Support

•Onsite consultation and coaching•Small grants

Incentives•Increased subsidy based on star rating

Outreach & Communications

•Statewide parent education campaign

All children are ready for kindergarten

F. Increased use of data to drive individual improvement plans

D. Improved quality among parents/practitioners receiving support

E. Increased understanding among parents of elements of high quality child care

H. Increased number/percentage of children care slots are with high quality programs/practitioners

A. Increased sustainability of high quality child care businesses participating

G. Increased number and percentage of parents who choose to enroll kinds inhigh quality care

C. More children participating in child care experience high quality

B. Increased school readiness among children participating in child care

5

8

1 2 3

4 6

7

10

F. Increased use of data to drive individual improvement plans

D. Improved quality among programs/practitioners receiving support

E. Increased understanding among parents of elements of high quality child care

H. Increased number/percentage of children care slots are with high quality programs/practitioners

A. Increased sustainability of high quality child care businesses participating

G. Increased number and percentage of parents who choose to enroll kids inhigh quality care

C. More children participating in child care experience high quality

B. Increased school readiness among children participating in child care

30,000 Foot Assumptions

11

Assumptions - What are some of the assumptions that are explicit or implicit in the Disarray QRIS plan? What is the basis for those assumptions? Are they valid or should they be revisited? Accountability -What outcomes are the team designing/implementing accountable for achieving? Do the strategies match the outcomes?

* Their plans seem to assume an aggressive data collection and analysis process* The role of their Quality Improvement Plans is not clear* I wonder whether they have thought through adequate dosage (time and intensity) to show results* It is not clear that there is a relationship between their quality improvement process and their quality standards* They are making a huge assumption that the data being collected is related to the quality standards adopted* Prior to outcome D, they need to address assessment of the effectiveness of the coaching provided as well as fidelity to their coaching model



D. Increased common understanding of definition of quality among state’s child care stakeholders

A. Increased understanding of current level of quality among existing providers

C. Increased use of data to drive individual improvement plans

B. Increased identification of opportunities to improve quality of existing programs

3

1

4

2

• Align quality standards with professional standards and early learning guidelines

• Review and revise quality standards periodically

• Align quality standards with professional standards and early learning guidelines

• Create data collection, analysis and reporting process/system and practices

• Assess and periodically monitor learning environment and interactions

• Analyze trends

• Select/ design assessment tools and processes

• Assign SODS rating

• Align standards with assessments and ratings tools

12

SET QUALITY STANDARDS ACCOUNTABILITY: CONDUCT RATINGS GATHER AND MANAGE DATA

D. Increased common understanding of definition of quality among state’s child care stakeholders

C. Increased use of data to drive individual improvement plans

B. Increased identification of opportunities to improve quality of existing programs

A. Increased understanding of current level of quality among existing providers


13

•What outcomes are the team designing/implementing accountable for achieving? Do the strategies match the outcomes? Are there other outcomes that would you suggest adding in order to describe the theory and show short-term accountabilities?•Any thoughts about Assumptions? What are some of the assumptions that are explicit or implicit in the Disarray QRIS plan? What is the basis for those assumptions? Are they valid or should they be revisited?

* The program is voluntary, yet it is expected to raise understanding of quality and actual quality across the board* It is not clear what the measure of "raise awareness about quality" might be (we discussed the bellwether evaluation method in which representative thought leaders are asked to describe high-quality child care with post-pilot results compared to pre-pilot results)* It is not clear from what the State of Disarray has been handed, what audience is intended to be the target of awareness raising about quality* Their situation raises the question of whether we are trying to change awareness (e.g. ability to describe quality) or change behavior (e.g. ability to create quality or purchase quality) in different situations


State of Disarray QRIS Pilot•Standards•Accountability•Program/Practitioner support•Financial incentives•Outreach and education

Expansion Strategies

Long-term Financial Strategies

Communications Outreach and Education

•Standards, ratings of programs•Parents, policymakers, public


D. Increased support of funding for QRIS among policymakers

E. Changed state norm related to quality expectations for child care

H. Increased school readiness among children participating in child care

A. More children participating in child care that exercises high quality

G. Increased understanding among parents and publicof quality care and role of QRIS

C. Increased school readiness among children participating in child care QRIS pilot providers

B. Increased school readiness among all children

8

1 2 3 4

5

7

I. Increased number/ percentage of participants participating in QRIS

6

14

D. Increased support of funding for QRIS among policymakers

E. Changed state norm related to quality expectations for child care

H. Increased school readiness among children participating in child care

A. More children participating in child care that exercises high quality

G. Increased understanding among parents and publicof quality care and role of QRIS

C. Increased school readiness among children participating in child care QRIS pilot providers

B. Increased school readiness among all children

I. Increased number/ percentage of participants participating in QRIS


15

What are some of the assumptions that are explicit or implicit in the Disarray QRIS plan? What is the basis for those assumptions? Are they valid or should they be revisited? How about those outcomes? Do they resonate with you? Would you suggest additional or different system-level outcomes?

* Our state evaluator has commented on the importance of qualitative data captured through interviews that will help us to design something that actually lifts all boats* Norms can be difficult to measure, but it is possible to assess changes in cultural attitudes

Discussion and Initial Thoughts

• We’d like to hear back from each state, if we could, with your initial reactions to today’s material. Share an “Aha”, an idea that you’d like to pursue, or do differently

16

For what outcomes IS it reasonable to hold QRIS accountable?

VA - We really need to formulate our assumptions and compare the infrastructure needed with what we have in place. This process helps us to see if our goals are realistic and what we will need to get there.NY - This points out that there are multiple levels of assessment (e.g. coaching fidelity--> coaching effectiveness --> teach interaction improvement, etc.) for us to frame outNJ - This makes clear that we need to focus more (and help others to do so) on interim outcomes for which we can be held accountable. Having these sample maps helps to clarify that and shows how we need to broaden our formulation of our assumptionsND - It is clear now how important specificity of the narrative and outcomes isMN - This process will help us to identify and articulate to others what we will be accountable forMA - We have parts and pieces. This helps to think about how to connect them and leverage the efforts of other initiatives and place accountable where appropriate

* Piloting in a variety of settings - relevance* Ensure exposure to what quality is and spur participation* Provision of consistent and high-quality coaching and support* Ensuring that funding and quality levels are connected

17

FeedbackWebinar Polling

POLL 1• How prepared do you feel for creating Theories of Action, Logic

Models and Outcome Maps for your state QRIS?0% - Not at All 83% - Somewhat 17% - Considerably 0% - Completely

18


POLL 2• Are you familiar with the recently published NCCIC

Compendium on QRIS Systems and Evaluations?17% - Not at All 33% - Somewhat 42% - Considerably 15% - Completely

19


POLL 3• Are you familiar with the RAND Report on QRIS Systems in

Five Pioneer States?8% - Not at All 54% - Somewhat 23% - Considerably 15% - Completely

20

Homework Assignment

• By Noon Pacific Time 11/16/10, please send your state’s response to the following questions to Heidi Brown at ORS [email protected] so that we can use it as the basis for conversation during the December 1st Learning Table:1. In thinking of the three levels of QRIS Outcome Maps for the

State of Disarray we discussed today, what level of outcome map is most useful for what your state is in planning? Please share some reasons why this is helpful.

2. When comparing the strategies and outcomes in the State of Disarray’s Outcome Maps to your own state’s efforts, which additional strategies and outcomes do your plans include and which are missing? If you are not thinking about changing your plans to accommodate these differences, what are the assumptions that underlie your decision?

21

• Next session:

December 1, 201010:30 AM Pacific 12:30 PM Central 1:30 PM

Eastern

• Additional feedback, send to [email protected]

Thank you for your participation!

Wrap Up

Documents

1 QRIS National Learning Network Learning Table Welcome! 1 “Strengthening Inquiry”