National Center on Child Care Quality Improvement QRIS
Standards Learning Table Session #3: Efficiency in Monitoring:
Streamlining Documentation
Slide 2
Introductions and Updates Introduce the state team (Name,
title, agency) AL, CA, CT, GA, HI, NV, OR, VI Describe what your
state team has been doing with regard to your QRIS since our last
call. It could be related to the homework or other points of
interest in your work. Share strategies your state is using to
bring these concepts and materials back to workgroups within the
state. (agendas, topics, etc.) If a certain resource or idea has
been particularly helpful, tell us about that. 2
Slide 3
Key Point Even a QRIS that appears simple can become complex
and expensive to administer unless steps are taken to streamline
the documentation procedures for standards and sources of
evidence.
Slide 4
Sources of Evidence For each standard you must: Clarify if/when
documentation is required Example: If youve already seen a source
of evidence in the past, do you need to see it each year? Specify
what documentation can be accepted to verify compliance.
Slide 5
Efficiency Opportunity: Current Assessment Tools as Source of
Evidence Some Program/Classroom Assessment tools measure the same
content. Some Program/Classroom Assessment tools measure criteria
included in a states QRIS. Thus, a QRIS could use an Assessment
tool such as ERS or PAS as the source of evidence.
Slide 6
Do the common tools measure the same concepts?
ECERS-RFCCERS-RCLASSPASBAS General Cognition Social & Emotional
Development Approaches to Learning Heath/Physical Development
Business Practices Family Involvement Internal Communication
Leadership/Management ?
Slide 7
Efficiency Opportunity: Self-Report What standards are most
appropriately verified by self-report? What are effective
procedures for validating self-reporting? Is random sampling
appropriate? What documentation needs to be available for
review?
Slide 8
Efficiency Opportunity: Automation How can automation
streamline the monitoring process? Links to data-bases for
licensing, registry, CACFP, subsidy, accreditation Electronic
scoring/reporting of ERS, CLASS, PAS/BAS Director portals or
on-line applications (in real time) that enable programs to upload
evidence Provider-focused platforms that include downloadable
tools/templates to support compliance
Slide 9
Automation: Learning Table States Results from both sessions:
Links to Registry: AR, DE, NH, OR, GA (OK, CA, PA developing) Links
to Licensing: DE, KY, NM, OK, TX, OR, PA Links to PreK Monitoring:
NM Links to Head Start Performance Review: AR, DE, OK On-line
Upload of Documentation: AR, NM
Slide 10
Case Study: Maine Michel Lahti, PhD University of Southern
Maine
Slide 11
QUALITY FOR ME THE BASICS Licensing compliance Membership in
MRTQ Registry Online application based upon a self-evaluation Once
the on-line application is submitted, the provider immediately
receives feedback from the Quality for ME system regarding the
anticipated Step level Portfolio of documentation (random) On-site
Observations (random)
Slide 12
The General Approach Web-based application Linkage to licensing
database and PD Registry Relieves burden for all applicants
Improves data quality in QRS application Feedback loop also
improves data quality in linked database Criteria cross-walked with
Accreditation criteria Self-report on remaining items About 50
specific questions if no Accreditation Reduced to just 5-10
questions depending on Accreditation Immediate feedback on how to
move to next step in each area Individual and aggregate reports
shared with R&R centers to facilitate technical assistance
Slide 13
QRS Step in Each of Eight Areas: compliance history/licensing
status learning environment/developmentally appropriate practice
program evaluation staffing and professional development
administrative policies and procedures parent/family involvement
family resources authentic assessment QRS Step in Each of Eight
Areas: compliance history/licensing status learning
environment/developmentally appropriate practice program evaluation
staffing and professional development administrative policies and
procedures parent/family involvement family resources authentic
assessment Criteria for achieving steps cross-walked with standards
for the following: NAEYC Accreditation NAEYC Candidacy National
Association of Family Child Care Providers Accreditation National
After School Association Accreditation American Montessori Society
Accreditation Head Start: Zero Non-compliance Issues at Last Review
/ All Non-compliance Issues at Last Federal Review Resolved
Criteria for achieving steps cross-walked with standards for the
following: NAEYC Accreditation NAEYC Candidacy National Association
of Family Child Care Providers Accreditation National After School
Association Accreditation American Montessori Society Accreditation
Head Start: Zero Non-compliance Issues at Last Review / All
Non-compliance Issues at Last Federal Review Resolved
Slide 14
Program Licensing MeDHHS, Augusta program license # contact
info capacity license status license expiration type of program
Maine Roads To Quality (Prof Dev Registry) Univ of Southern Maine,
Portland provider ID provider education provider training record
license # of program where provider employed Quality Rating System
Univ of Maine, Orono program license # self-reported data
calculated data Maine Roads To Quality (Prof Dev Registry) Univ of
Southern Maine, Portland program license # accreditation Key Data
Linkages
Slide 15
Improves Data Quality at Linked Databases
Slide 16
Immediate Feedback to Applicant
Slide 17
(specific recommendations for each of 8 areas)
Slide 18
Data Usage Monitor Enrollments and Characteristics of Programs
ERS Scores Focus on Areas of Strength and Improvement Monitor
Program Progress through Step Levels Monitor Supports to Programs
Infrastructure for Evaluation Projects: Comparing QRIS to non-QRIS
Sites Investigate QRIS Standards: Use of Child Level Assessments
Validation Study
Slide 19
Lessons Learned from Maine Intention is to Build a System, an
Infrastructure to Help Align ECE Programming Develop Working
Partnerships with State Program Administrators and University
Research Staff System Operation Requires Ongoing Attention - Keep
it Valid and Reliable Importance of Translating Data from QRIS
Monitoring into Information for Decision-making
Slide 20
BENEFITS TO JOINING QUALITY FOR ME Ability to accept Child Care
Subsidy and receive a payment differential based upon Step Level
Assistance in paying for Accreditation fees and cohort supports
(some facility improvement grants) On-site technical assistance
Scholarships to pursue early childhood education degrees Tax
credits for parents and providers
Slide 21
Automation of QRIS Implementation Results from both sessions
and other states included: WELS (FL, NY, MS) MOSAIC (MI, CA)
BRANAGH (LA) State - Developed Systems (AZ, ME, GA, PA)
Slide 22
Georgias Online System Background Design - 2011-2012 Launched
1/2012 Equal emphasis on Process Quality (ERS) and Structural
Quality (Program Portfolio In-house design and development of
online system to manage all of Quality Rated from process to data
Application Training/Technical Assistance Registration to tracking
Portfolio Submission CQI Plans Incentives Management Resources
Reports and Data Communication
Slide 23
Quality Rated Components
Slide 24
Research Questions Data Dictionary Validation and research
guided development of online system With TA support from FPG
Developed logic model Developed validation and evaluation model
Created data dictionary Created reports
Slide 25
Validation Plan by Phase
Slide 26
Evaluation Plan Type of evaluation questions Year 1 2012-2013
Year 2 2013-2014 Year 3 2014-2015 Year 4 2015-2016 Year 5 2016-2017
Participationxxxxx Retentionxxxxx Motivation xxx Quality Supports
(TA, Training, Incentives, Tiered Reimbursement, Bonus Packages)
xxx DECAL Resources and Processes xxxx Statewide Quality
Improvement x
Slide 27
Ongoing Validation and EvaluationOngoing Validation and
Evaluation
Slide 28
Online Site Users Quality Rated staff Technical assistance
staff Resource and referral agencies Programs enrolled in QR
Incentive partners Research team Parents in 2013 will see
levels
Slide 29
Quick Tour of the Site
Slide 30
Lessons Learned in Georgia Keep it simple Resources make all of
the difference Transparency INVOLVE THE RESEARCHERS
Slide 31
Web-Based Supports for Providers The Raise Quality Tab on
ECESharedResources.org:
http://national.ecesharedresources.net/index/ SharedSource PA,
Child Care Tennessee, New Mexico Early Learning Alliance Oregon
Maine
Slide 32
Efficiency Opportunity: Multi-Site Centers How is documentation
streamlined for multi- site centers? What information can be
gathered from the central office? What must be gathered at each
site? What standards might be revised given a multi-site management
framework?
Slide 33
QRIS Administration with Multi-Site Centers: State Examples New
Mexico Verify documentation at central office Oklahoma Head Start
participation Georgia Cohort structure
Slide 34
Verification: Effective and Efficient? Standards Think Tank
Participants thought the most effective and efficient verification
methods were: Objective Third Party Observation/Assessment
Electronic Link to Licensing, Registry or other Official Database
Self-Report with Verification of Random Sample
Slide 35
Questions, Reflections, Comments?
Slide 36
Next Session - Homework 1. When you consider your states
initial QRIS standards or in the early stages of implementation,
what types of considerations are your team discussing? Ease of
administration of the QRIS Standards that provide administrative or
research data (e.g., must enroll in the state Professional
Development Registry) Ease of participation for early care and
education programs in QRIS (e.g., Do the standards begin very low
to entice enrollment? How rigorous is the highest level?) Research
base for the standards Standards that address emerging issues
(e.g., diversity, child assessment, reflective practice) Alternate
pathways for various provider types (under what circumstances for
which types of providers) Other 2. What data or research did you
use to guide your selection of QRIS standards and what type of data
are you collecting to guide future revisions? Information from
participants in the QRIS (programs, providers, parents) Data from
your QRIS management system State Research National Research
Other
