The Importance of Achieving Quality Outcomes - An Overview Jennifer Johnson July 8, 2013

The Importance of Achieving Quality Outcomes - An

Overview Jennifer Johnson

July 8, 2013

OVERVIEW OF FEDERAL ACCOUNTABILITY SYSTEM

2

3

Long-Term Move Towards Performance

Performance management is not a new phenomenon—over 50 years of work in the making to link resources with results

– Budget Accounting Procedures Act (BAPA) of 1950– Planning-Programming-Budgeting System (PPBS),

1965-1971– Management by Objectives (MBO) 1973-1974– Zero-Based Budgeting (ZBB), 1977-1981

GPRA

• Originally passed in 1993

• Government Performance and Results Act (GPRA) Modernization Act of 2010 (GPRAMA)– GPRA evolved from the original development of

performance measures to a more mature, modernized and important law of government that specifies requirements and measures, as well as processes for ongoing monitoring of measures.

4

GPRAMA• Continues three agency-level products from GPRA 1993, but

with changes

• Establishes new products and processes that focus on goal-setting and performance measurement in policy areas that cut across agencies

• Brings attention to using goals and measures during policy implementation

• Increases reporting on the Internet

• Requires individuals to be responsible for some goals and management tasks.

5

Potential Benefits of GPRAMA• Adopting a more coordinated and crosscutting

approach to achieving meaningful results. – GPRAMA could help inform reexamination or restructuring

efforts and lead to more efficient and economical service delivery in overlapping program areas by identifying the various agencies and federal activities that contribute to crosscutting outcomes.

– These program areas could include multiple employment and training programs or numerous teacher quality initiatives, among others.

6

Potential Benefits of GPRAMA

• Addressing weaknesses in major management functions. – Agencies need more effective management

capabilities to better implement their programs and policies.

– GPRAMA requires long-term goals to improve management functions in five key areas: financial, human capital, information technology, procurement and acquisition, and real property.

7

Potential Benefits of GPRAMA• Ensuring performance information is both useful and used in

decision making. – Agencies need to consider the differing needs of various stakeholders,

including Congress, to ensure that performance information will be both useful and used.

– For performance information to be useful, it must be complete, accurate, valid, timely, and easy to use.

– Yet decision makers often do not have the quality performance information they need to improve results.

– To help address this need, GPRAMA requires (1) disclosure of information about accuracy and validity, (2) data on crosscutting areas, and (3) quarterly reporting on priority goals on a publicly available Web site.

8


• Instilling sustained leadership commitment and accountability for achieving results.– GPRAMA assigns responsibilities to a Chief

Operating Officer and Performance Improvement Officer in each agency to improve agency management and performance.

9


• Engaging Congress in identifying management and performance issues to address. – GPRAMA significantly enhances requirements

for agencies to consult with Congress.

10

DD Council GPRA Measures

• Increase the percentage of individuals with developmental disabilities reached by the Councils who are independent, self-sufficient and integrated into the community. (Outcome)

11


• Increase the number of individuals with developmental disabilities reached by the Councils who are independent, self-sufficient and integrated into the community per $1,000 of federal funding to the Councils. (Efficiency)

12


• Number of individuals with developmental disabilities reached by the Councils who are independent, self-sufficient and integrated into the community. (Output)

• Number of all individuals trained by the Councils. (Output)

13

How are DD Council GPRA Measures Used?

• AIDD analyzes the data for trends and results

• ACL includes the results in budget justifications that goes to the Office of Management and Budget (OMB) and then to Congress

14

What happens to the GPRA Measures?

15

AIDD ACL

HHS

Congress

OMB

WHY MEASURE?

16

What gets measured

gets done.

–Peter Drucker

17

Why Measure?

To Plan?

To Comply?

To Manage?

To Optimize?

To Innovate?

18

Performance Management

• Translates a mission into reality and evaluates the results to all stakeholders

• Transparency of Performance to:– Donors– Elected Leaders– Senior Management– Oversight Entities– Employees– Customers– Partners

19

Performance Management• Strategic Level

– Measure Progress on Issues– Define & Validate Policy Strategies– Enhance Stakeholder Satisfaction and Support

• Operational Level– Drive Change to Implement Organizational Strategies– Ensure Compliance – Achieve Efficiencies– Improve Cycle Time

• Individual Level– Improved Morale/Retention– Achieve Clarity of Responsibilities

20

Approaches to Performance Management

• Program Evaluation

• Performance Measurement

21

22

Effective Performance Measures are SMART

S PECIFIC

M EASUREABLE

A CCOUNTABLE

R ESULTS-ORIENTED (#1)

T IME-BOUND

23

Applying SMART

End-Outcome: Reduce smoking-related deaths, illness and costs

Intermediate Outcome: “Reduce the number of new youth smokers (10-18) by 2% each year”

– Results-oriented: Youth smoking is where you can stop the habit before it takes hold and has a lasting health impact

– Specific: “number of new youth smokers (10-18)”

– Accountable: You have the ability to make it happen

– Measurable: “reduce by 2%”

– Time-Bound: “per year”

24

Performance Measure Selection Criteria

Meaningful Reports tangible and significant accomplishmentsagainst objectives; useful for decision makers andother stakeholders

Clear Easily understand by managers, partners, otherstakeholders; tells clear story

Legitimate Accepted or legitimated by those who must usethe data

Consistent Clear definition and data collection methodologyacross populations

Reliable Captures what it purports to measure in anunbiased fashion

Granular Able to detect performance movement

Relevant Does not become obsolete too early: sets apattern or baseline of performance

Performance Measure Selection Criteria

TechnicallyAdequate

Available data meets technical adequacy standards suchas accuracy, validity, timeliness

Responsible Does not have unintended and undesirableconsequences

Actionable Indicates what is good or bad, driving desired behaviorand the timing of action

Accountable Related to direct action or influence of an accountableand attributable entity

Balanced One of set of measures providing a clear picture of the fullrange of performance

Vital Measures are mutually exclusive and collectively exhaustive

Feasible Reasonable cost and accessibility of data that is notalready collected

Culture shift

FromFrom• These measures are These measures are

draining valuable draining valuable resources and are a data resources and are a data burdenburden

• I can’t measure my I can’t measure my outcomes I can only outcomes I can only measure activitiesmeasure activities

• I need these measures I need these measures because my funders feel it because my funders feel it is importantis important

• You can’t measure my You can’t measure my program.program.

To To • We are committed to We are committed to

tracking measures that tracking measures that matter most.matter most.

• We are accountable for We are accountable for delivering our outputs and delivering our outputs and our intermediate outcomes.our intermediate outcomes.

• We are responsible for our We are responsible for our end outcomes.end outcomes.

Performance Measures Definitions

• Performance Measures: Indicators, stats, metrics used to gauge program performance

• Target: Quantifiable characteristic that communicates to what extent a program must accomplish a performance measure

• Outcome Measures: Intended result of carrying out a program. Define an event or condition external that is a direct impact the public.

• Output Measures: Describes the level of activity that will be provided over a period of time

• Efficiency Measures: Measures that capture the skillfulness in executing programs, implementing activities, and achieving results

Current state of measurement

• Too Many Measures

• Wrong Kinds of Measures– Too process and activity oriented

– No clearly defined “Logic Model”

– No measures of strategy

– Few measures of end outcome

• Diluting Measures– Measuring only the things you can count rather

than things that are strategically important

Not everything that can be counted counts and not everything that counts can be counted. - Albert Einstein

IDENTIFYING CHARACTERISTICS OF EFFECTIVE PERFORMANCE MANAGEMENT SYSTEMS

8 Critical Success Factors for Effective Performance Management Systems

1. Defining and Aligning to Enterprise Strategy

2. Developing Meaningful Performance Measures

3. Increasing Data Availability

4. Maximizing Data Integrity

5. Enhancing Performance Reporting

6. Improving Evaluation and Analysis

7. Achieving Performance Integration

8. Driving Decision-Making

#1 Defining & Aligning to Enterprise Strategy

1.1 Has clearly defined its mission, vision and values

1.2 Has specific strategies in place to achieve organizational results (based on a SWOT or other strategic landscape analysis)

1.3 All structures (divisions, support functions) are fully aligned with enterprise-wide strategies

1.4 A formal strategic plan is clearly communicated to all employees at all levels of the organization

#2 Developing Meaningful Performance Measures

2.1 Reliable measurement and reporting on Outcomes

2.2 Reliable measurement and reporting on Strategies

2.3 Organizational process metrics (Quality, Cycle Time, Efficiency)

2.4 Goals and measures enjoy support and buy-in from internal and external stakeholders

#3 Increasing Data Availability

3.1 Data sources are identified and readily accessible

3.2 Data burden is worth the information gleaned

#4 Maximizing Data Integrity

4.1 Data is collected, managed, and analyzed in a uniform and consistent manner

4.2 Data is validated or verified through sampling or independent means

#5 Enhancing Performance Reporting

5.1 Internal reporting produces information for frontline managers and senior decision-makers on a “real time” basis.

5.2 Has a reporting system that produces comprehensive performance reports that include measures, analysis, trends, suggestions for improvement

#6 Improving Evaluation and Analysis

6.1 For process measures, benchmarks and service levels are evaluated (1-2 year cycles)

6.2 For outcome and strategy measures, program performance is evaluated for “cause-effect” (2-5 year cycles)

#7 Achieving Performance Integration

7.1 INTERNAL Integration: Support services’ contributions (HR, IT, Finance, etc.) to program performance is documented and managed

7.2 EXTERNAL Integration: Performance contributions of multiple contributors in samemeasurement area are tracked and compared

#8 Driving Decision-Making

8.1 Budgets and investments are made based on clear contributions to performance

8.2 Supply chain partners are held accountable for products and services

8.3 Employee bonuses and pay increases are linked to individual performance evaluations.

Understanding Logic Models

What is a logic model?1. Logical chain of events providing blueprint for mission

achievement

2. Graphic representation that illustrates the rationale behind a program or organization

3. Depicts causal relationships between activities, strategies, and end results

4. Contains goals and performance measures

5. Integrates various program activities into a cohesive whole

6. Vehicle for dialogue, planning, program management and evaluation

What does a logic model look like?

• Graphic display of boxes and arrows; vertical or horizontal

– Relationships, linkages

• Any shape

– Circular, dynamic

– Cultural adaptations, storyboards

• Level of detail

– Simple

– Complex

• Multiple models

Logic modeling is based on mapping and defining linkages between what we do and why we do it .

I Work Out for One Hour Each Day

I Will Burn More Calories Than I Consume

Lose Fat and Build Muscle

Improve My Looks and Health

Have Better Image,Feel Better & Live Longer

IF THEN IF THEN IF THEN IF THEN

INPUTS OUTPUTS OUTCOMES

Series of If-Then Relationships

Assumptions: improving looks = better self image

Factors: Health History

InputsPeople and resources required to achieve outcomes

Activities/OutputsWhat the inputs produce

Immediate and Intermediate OutcomesChanges required to achieve end outcome

End OutcomeEnd goal or ultimate benefit

Clarifying the terms

Assumptions: beliefs or evidence that supports your IF-THEN logic

Factors: external influences beyond control that effect IF-THEN relationships

Inputs Activities Outputs End Outcomes

Most logic models incorporate the following elements.

Intermediate Outcomes

Attitudes Behaviors Conditions

WHY?HOW

EFFECTCONTROL

Drivers Inputs Activities OutputsIntermediate Outcomes

EndOutcomes

• Resources used to support activities • Not performance measures• System integration: Link inputs through

process to goals and outcomes in:–Budget requests–Human resource plans–Information technology plans

• Resources used to support activities • Not performance measures• System integration: Link inputs through

process to goals and outcomes in:–Budget requests–Human resource plans–Information technology plans


EndOutcomes

• Processes and roles• What the program does• Subject of on-going process

improvement and strategy change• System integration through linkages to:

–Stakeholder satisfaction–Assessment quality–Desired outcomes–Efficiency measures

• Processes and roles• What the program does• Subject of on-going process

improvement and strategy change• System integration through linkages to:

–Stakeholder satisfaction–Assessment quality–Desired outcomes–Efficiency measures


End Outcomes

•Products and services delivered (e.g. grants, audits, research studies, impact assessments, etc.)• Indicate strategy deployment• Foundation step for attainment of all types of outcomes• Often measured by low-level outcome types

-Process vital signs-Customer satisfaction

• Good source of short-term, readily available results• 12 months-1 year

•Products and services delivered (e.g. grants, audits, research studies, impact assessments, etc.)• Indicate strategy deployment• Foundation step for attainment of all types of outcomes• Often measured by low-level outcome types

-Process vital signs-Customer satisfaction

• Good source of short-term, readily available results• 12 months-1 year


EndOutcomes

• Show cross-agency/program accountabilities• Can be one or several in number• Often measured by attitudes, behaviors and

conditions• Can show short- to medium-term change • 1-5 years

• Show cross-agency/program accountabilities• Can be one or several in number• Often measured by attitudes, behaviors and

conditions• Can show short- to medium-term change • 1-5 years

Connecting strategies, intermediate outcomes and measures

U.S. Department of Labor Women’s Bureau

Strategy: Provide training in high-growth, demand-driven occupations to women

Intermediate Outcome: Increase hard skills in high-growth, demand-driven occupations for participants

Intermediate Outcome Performance Measure: % of women participants who successfully complete training, education or certification for high-growth, demand-driven occupations

Intermediate Outcomes


End Outcomes

• Shows ultimate benefit to tax payer• Often measured by long-term indicators

–Changes in economic, policy conditions • Captured in the strategic plan as end

goals• 5-10 years

• Shows ultimate benefit to tax payer• Often measured by long-term indicators

–Changes in economic, policy conditions • Captured in the strategic plan as end

goals• 5-10 years

Connecting mission, goals, end outcomes and measures

U.S. Department of Labor Women’s Bureau

Mission: Improve the status of wage earning women, improve their working conditions, increase their efficiency, and advance their opportunities for profitable employment.

Goal: Improve status of working women through better jobs.

End Outcome: Increase women’s employment in high-growth, demand-driven occupations.

End Outcome Performance Measure: % of women participants who find employment in high-growth, demand-driven occupation

End Outcomes

DriversDrivers InputsInputs ActivitiesActivities OutputsOutputs Intermediate OutcomesIntermediate Outcomes

EndOutcomesEndOutcomes

OverallOverall

Relating Logic Models to Strategic Plan Elements

Programs

Projects, tasks or initiatives designed to contribute to end outcomes

Outcome Goals

Developed from statute based mission$/FTE

Internal/External Environment

Statutory Authority

Based on strategic assessment

Mission

Values

Performance

Measures

Strategies

Changes in attitudes, behaviors or conditions required to achieve outcome goals

AIDD PERFORMANCE MANAGEMENT SYSTEM

53

Where have we been and where are we now?

5-YEAR PLAN

MTARS

Indicators of Progress(PPRs)

The system formerly known as

PART

DDPIE

PPRs

GPRA(PPRs)

ACCOUNTABILITY

DD Act Requirements

• Sec. 104. PROGRAM ACCOUNTABILITY.— – IN GENERAL.—In order to monitor entities that received

funds under this Act to carry out activities under subtitles B, C, and D and determine the extent to which the entities have been responsive to the purpose of this title and have taken actions consistent with the policy described in section 101(c), the Secretary shall develop and implement an accountability process as described in this subsection, with respect to activities conducted after October 1, 2001.

55

DD Act Requirements

• Sec. 104. PROGRAM ACCOUNTABILITY.— – AREAS OF EMPHASIS.—The Secretary shall develop a process

for identifying and reporting…on progress achieved through advocacy, capacity building, and systemic change activities, undertaken by the entities described in paragraph (1), that resulted in individuals with developmental disabilities and their families participating in the design of and having access to needed community services, individualized supports, and other forms of assistance that promote self-determination, independence, productivity, and integration and inclusion in all facets of community life….in the areas of emphasis.

56

DD Act Requirements

• Sec. 104. PROGRAM ACCOUNTABILITY.— – INDICATORS OF PROGRESS.—

• At a minimum, the indicators of progress shall be used to describe and measure—

– the satisfaction of individuals with developmental disabilities with the advocacy, capacity building, and systemic change activities provided under subtitles B, C, and D;

57

DD Act Requirements• Sec. 104. PROGRAM ACCOUNTABILITY.—

– the extent to which the advocacy, capacity building, and systemic change activities…result in improvements in the ability of individuals with developmental disabilities to—

• make choices and exert control over the type, intensity, and timing of services, supports, and assistance that the individuals have used;

• to participate in the full range of community life with persons of the individuals' choice; and

• to access services, supports, and assistance in a manner that ensures that such an individual is free from abuse, neglect, sexual and financial exploitation, violation of legal and human rights, and the inappropriate use of restraints and seclusion

58

DD Act Requirements– the extent to which the entities described in paragraph (1)

collaborate with each other to achieve the purpose of this title and the policy described in section 101(c).

59

DD Act Requirements

• Sec. 105. Reports of the Secretary. [42 USC 15005]– At least once every 2 years, the Secretary, using information

submitted in the reports and information required under subtitles B, C, D, and E, shall prepare and submit to the President, Congress, and the National Council on Disability, a report that describes the goals and outcomes of programs supported under subtitles B, C, D, and E. In preparing the report, the Secretary shall provide—

60

DD Act Requirements• Meaningful examples of how Councils, P&As, UCEDDs, and PNs project

respectively— – have undertaken coordinated activities with each other;

– have enhanced the ability of individuals with developmental disabilities and their families to participate in the design of and have access to needed community services, individualized supports, and other forms of assistance that promote self-determination, independence, productivity, and integration and inclusion in all facets of community life;

– have brought about advocacy, capacity building, and systemic change activities (including policy reform), and other actions on behalf of individuals with developmental disabilities and their families, including individuals who are traditionally unserved or underserved, particularly individuals who are members of ethnic and racial minority groups and individuals from underserved geographic areas; and

– have brought about advocacy, capacity building, and systemic change activities that affect individuals with disabilities other than individuals with developmental disabilities;

61

DD Act Requirements• information on the extent to which programs authorized under

this title have addressed— – protecting individuals with developmental disabilities from abuse, neglect,

sexual and financial exploitation, and violations of legal and human rights, so that those individuals are at no greater risk of harm than other persons in the general population; and

– reports of deaths of and serious injuries to individuals with developmental disabilities; and

• a summary of any incidents of noncompliance of the programs authorized under this title with the provisions of this title, and corrections made or actions taken to obtain compliance.

62

GPRA(PPRs)

PPRs

Cross Program Performance Measures

Indicators of Progress(PPRs)

Revised MTARS

5-YEAR PLAN

5-YEAR PPR

What does the future hold?

Future

• Integrated Performance Management System– Program Evaluation

– Performance Measurement

– Monitoring System

Future• MTARS (Monitoring and Technical Assistance

Review System)– Revising system used to monitor grantees

– Focus on compliance and outcomes

• PPR & Indicators of Progress/Performance Measures – Changes to the indicators of progress based on

recommendations from Workgroup and feedback from Network

Future

• 5 Year State Plan template revised– Strategic Plan with measureable goals– Incorporate evaluation plan and logic model– Peer review process– Strengthen connection to evaluation by tying

changes/revisions in plan to PPR

66

PPR Revisions

• Implemented hybrid report using new PPR template

• Still working on performance measures– Individual Advocacy– Systems Change– Funds Leveraged NEW!

67

Workgroup Members

• Council Representatives:– Anna Lobosco, NY– Graham Mulholland, PA– Holly Riddle, NC– Wanda Willis, TN– Vendella Collins, MI– Joanna Cordry, TX– Ed Holen, WA

•

• AIDD Representatives:– Jennifer Johnson

– Sara Newell, AIDD Program Specialist

• ITACC Representatives:– Sheryl Matney

– Hillary Spears

mailto:[email protected]

Revised Performance Measures June – August, 2012: Sought feedback from

Councils

September 2012 – February 2013: Worked in sub-groups to focus on making changes to two performance measures (systems change and individual advocacy) based on Council feedback

Revised Performance Measures

• January, 2013: Council submit “hybrid” report• March, 2013 – June, 2013:

– Full workgroup reviewed changes and provided feedback

– Discussed third measure of funds leveraged– Discussed pilot process– Internal meetings with ACL to solicit feedback

70


What we need your reactions, ideas, your reactions, ideas, feedbackfeedback

1. What do you like about the draft revised performance measures?

2. What are your concerns with the draft performance measures?

3. Where do you need more information?

71

Opportunities to Provide Feedback

• Session at TA Institute from 4:30 – 5:30

• Participate in webinar on July 22, 2013 from 3:30 – 4:30 EST

• Submit comments/feedback in writing by July 31, 2013 to [email protected]

72

mailto:[email protected]


• AIDD plans to conduct a pilot of the draft performance measures to determine if the measures: – Are feasible, reliable and valid– Need to be revised to strengthen feasibility,

reliability and validity– Identify and develop guidance for collecting the

data

73

Timeline for PilotFY 2014 October 1, 2013 – September 30, 2014PPR due: January 1, 2015

FY 2015October 1, 2014 – September 30, 2015PPR due: January 1, 2016



Pilot Phase 1: Alignment Use year to set up and/or align grants/contracts and data systems in preparation for collecting data on new performance measuresFY 2014 PPR – all Councils submit data using current performance measures

Pilot Phase 2: Data CollectionUse year for pilot group to collect data on new performance measures Rest of Councils collect data on current measuresFY 2015 PPR – pilot group submit data on new performance measures /rest of Councils submit using current measures All Councils start to align data systems and grant/contracts

Pilot Phase 3: Finalize AIDD finalizes measures based on pilotPilot group continues to collect data on new measures Rest of Councils collect data on current measures but finalize preparation to begin data collection in FY 2017FY 2016 - pilot group submit data on new performance measures /rest of Councils submit using current measures August 15, 2016 – all Councils submit new State Plans using new performance measures

New performance measures implemented

FY 2017 PPR - All Councils submit data on new performance measures

74

Pilot Group Commitment• Participate in pilot starting October 1, 2013

• Commit staff time to implement changes to data collection, reporting and grants/contracts systems to be able to test out new performance measures

• Document process to identify what is working and what is not working

75

Pilot Group Commitment• Participate in pilot group meetings to:

– Provide feedback on what is working/not working

– Recommend and review proposed changes

– Test out changes

– Assist with developing and revising instructions and forms for collecting data

– Assist with identifying and developing procedures and systems needed to collect data

– Provide peer to peer support and training as requested

76

Pilot Group Commitment

• It is anticipated that the pilot group will meet at least quarterly and as often as needed. The pilot group could meet as frequently as monthly if it is necessary

• We anticipate that all meetings will be conducted remotely and that travel will not be required

• More information to come in July!

77

New Cross Program Measure

• AIDD is exploring creating “super measures” for the DD Network programs

• These measure could replace the program specific GPRA measures in the PPRs

MTARS Streamlining Project

• In 2012, OPS engaged in a project to evaluate and revise the MTARS

• Phase 1 of the project conducted January – April, 2012

• Waiting for PPR data to assess effectiveness of Tier 1 and to further develop Tier 2

MTARS Streamlining – Phase 1 Goals

• Review and evaluate current AIDD MTARS model and materials with a particular focus on examining the relationship of monitoring, outcomes and technical assistance

• Review and evaluate other federal monitoring models

• Review and evaluate materials that could inform our monitoring process

80

MTARS Streamlining – Phase 1 Goals

• Conduct focus group sessions with key stakeholders

• Analyze results and develop recommendations for a revised monitoring system, that will inform Phase 2 of the project

81

MTARS Streamlining – Phase 1 Findings

Strengths of MTARS– Being On-site

– Team Composition

– Checklists

– Process and Feedback

Weaknesses of MTARS• Cumbersome

• Efficacy in addressing non-compliance/network collaboration

• Lack of follow up

• Not systematic

• Frequency

• Lack of focus on outcomes

• Lack of clarity in scope of work

• Combining TA and monitoring

• Length of time to produce report82

MTARS Streamlining – Phase 1 Recommendations

1. Rebrand the MTARS

2. Utilize a Tiered Review System– Tier one: annual compliance and outcome review of

PPRs/Plans

– Tier two: periodic in-depth desk audits of compliance and an outcome review conducted by one federal staff person and two peers, similar to the current MTARS team structure.

– Tier three: on-site review for red flag programs conducted by federal staff and peers as needed.

83

MTARS Streamlining – Phase 1 Recommendations

3. Have the TA Providers Conduct Organizational Reviews as Part of the Monitoring Process

4. Use Findings as the Foundation for Individual TA and National TA Plans

5. Have Integrated Compliance and Outcomes Review Tools in a Data Management System

6. Focus the Report on Information, Ensure Consistency in Format and Improve Publication Timeline

84

MTARS Streamlining – Phase 2

• Formed a workgroup of grantees, organizational representatives, and AIDD staff to finalize the revised model.

• Program teams established (DDC, UCEDD,

P&A) to develop a framework and the procedures for Tier 1 and Tier 2 review processes.

85


• DDC Workgroup members:– Wanda Willis, TN– Kathy McAllister, FL – Brian Cox, MD– Claire Mantonya, UT

86


• Program teams are developing the review tools:– Conducting analyses of the DD Act and program

regulation requirements and current PPRs and Plan templates for items related to compliance that can be reviewed on an annual basis.

– Identifying items that need to be added to templates.

87

MTARS Streamlining – Phase 2– Develop procedures for annual and periodic outcomes

reviews

– Determining how organizational reviews may be conducted

– Develop the criteria for reviewing the programs in Tier 1 and Tier 2, using existing tools as much as possible.

88


• Other activities:– Explore having a data management system– Conduct pilots in 2013– Revise the guidance– Continue to roll-out information to grantees– Conduct training as needed– Determine resources (staffing and funding)

89


• DD Council activities:– Collaborated with workgroup to identify what gets

reviewed in Tier 1 and Tier 2– Developed Tier 1 PPR Review tool– Piloted Tier 1 PPR Review tool March, 2013– Once completed the pilot of the Tier 1 PPR

review tool, will continue working on Tier 2 review tool

90

Administration on Intellectual and Developmental DisabilitiesAdministration for Community Living

U.S. Department of Health and Human Services

One Massachusetts Avenue, NWWashington, DC 20001

(p) 202-690-6590(f) 202-205-8037

AIDD website: http://transition.acf.hhs.gov/programs/addACL website: http://www.hhs.gov/acl

http://transition.acf.hhs.gov/programs/add

http://www.hhs.gov/acl

Documents

The Importance of Achieving Quality Outcomes - An Overview Jennifer Johnson July 8, 2013