ROMA Next Generation CSBG Annual Report …...Webinar Series Objectives Understand what ROMA Next...

ROMA NEXT GENERATIONCSBG Annual Report

Briefing Webinars

June 29, July 6 and July 142:00-3:30 EDT

1

WELCOME

PRESENTERSOffice of Community Services (OCS)Jeannie Chaffin, Director

NASCSPJenae Bjelland, Executive DirectorJackie Orr, CSBG State Assistance DirectorLauren Cook, CSBG Team LeadRae Tamblyn, Research and Communications Analyst

Association of Nationally Certified ROMA TrainersBarbara Mooney, Director

Federal Consulting GroupKate Blunt, Consultant

3

Webinar Series• ROMA NG Update/Overview—recorded webinar posted

on the NASCSP website• Overview of the State Administration Module—

recorded webinar posted on the NASCSP website• Module 3 (Community-Level): June 29, 2:00-3:30pm EDT• Module 4 (Individual/Family-Level Module: July 6, 2:00-

3:30pm EDT• Module 1 and 2 (Administrative & Agency Fiscal/

Capacity Modules): July 14, 2:00-3:30pm EDT4

Webinar Series Objectives Understand what ROMA Next Generation is

all about and the role the Annual Report plays.

Understand the specifics of what is being proposed in the new Annual Report.

Learn what is needed from you during 60-day comment period.

5

Today’s Agenda• ROMA Next Generation Overview:

Building on the ROMA Foundation• Draft Annual ReportCurrent StatusModule OverviewClarifications and Changes Made

• Questions and Answers• Next Steps: What is Needed Now

6

BUILDING ON THE ROMA FOUNDATION

7

ROMA Next Generation: The Goal

Move ROMA to a new level that will move CSBG forward and strengthen Community Action over the next 5-10 years. Just as ROMA 1.0 served Community Action well for over twenty years, ROMA Next Generation will move the field at all levels to increase impact.

Where We Want To BeOCS, States and CAAs use performance data

to achieve greater stability and economic security for families and communities.

Each level of the CSBG Network– executes the full ROMA cycle– uses data on people, services, and

outcomes for decision making

9

Where We Want To BeThe CSBG Network complies with the GPRA

Modernization Act of 2010:

“Our goal is to create a performance management framework that encourages good management and innovation without fear of penalty for failing to achieve every ambitious target that has been set but with heightened pressure to achieve breakthrough gains on priorities.”

10

ROMA Next GenerationFocuses on integrating information on the

people served, services and strategies provided, and the outcomes achieved by eligible entities. Is a system for continuous quality improvement

focused on creating learning organizations at all levels of the CSBG network.Requires attention to the capacity of state and

local data systems and support.

12

ROMA NG and the ROMA CYCLE

13

AssessmentAssess

community needs and resources

What does data tell us about the needs in our specified community? What identified needs can we reasonably address either independently or in partnership with other organizations?

ROMA NG and the ROMA CYCLE

14

PlanningUse agency

mission, TOC, and assessment to

support services/ initiatives

How do the Core Principles in the Theory of Change help guide our

agency strategic plan and community action plan?

How will we strategically respond as an agency to

individual, family, and community needs? What

impact do we want to have? What are our performance targets? What services and

initiatives do we select?

ROMA NG and the ROMA CYCLE

15

ImplementationImplement services

and initiatives (strategies) to

produce results

How will we implement our unique combination

of Services and Initiatives? And for what

Outcomes (NPIs)?

Do we have a process for observing progress and

using data to make changes as needed?

Do we need to make changes to our services and

initiatives?

ROMA NG and the ROMA CYCLE

16

Achievement of Results

Observe and report outcomes

and progress

What Outcomes are observed and measured as Community and Family level NPIs? How do the Outcomes we are tracking move us towards our agency goalsin our local TOC?

How do specific Services and Initiatives (Strategies) help

support and produce Outcomes to respond to the needs of our

community?

ROMA NG and the ROMA CYCLE

17

EvaluationAnalyze data, compare with benchmarks

Did the Outcomesachieved address the

needs identified in the assessment phase? How

well did we track our success? Did we meet our

targets?

What has changed for the people we served? What has changed for our community? What Outcomes were achieved and for whom? WhatServices and Initiatives contributed to achieving the Outcomes? What can we improve to better respond to local needs?

ROMA Next Generation

ROMA Next Generation also builds upon Organizational Standards and the State and Federal Accountability Measures to help achieve the full potential of CSBG.

18

ROMA Next Generation

ROMA Next Generation brings the assumptions and concepts articulated in the ROMA Cycle and the National Theory of Change (TOC) together and the results are reflected in the revised CSBG Annual Report.

CSBG ANNUAL REPORTCURRENT STATUS

20

ROMA NG CLEARANCE PROCESS

Three Steps1. Initial Feedback Period2. Formal 60-day OMB Comment Period3. Formal 30-day OMB Comment Period

21

Initial Feedback Period CompletedWorking together NASCSP/OCS….– Compiled the Survey data (on NASCSP’s website)– Modified the draft Annual Report and identified

areas needing clarification– Revised the Annual Report forms – Developed some supplemental documents to

support your review of the CSBG Annual Report (on NASCSP’s website)

www.nascsp.org22

Federal Register Notice posted on June 16, 2016: 60-day comment period runs through August 15, 2016Dear Colleague Letter published on June 17, 2016Formal 30 Day Comment Period: September

23

COMMUNITY LEVEL EFFORTS

24

Community-Level EffortsTells the story of the community-level impact of CSBG.Creates a common set of measures to track the CSBG

Network’s performance with respect to the impact of strategies provided at the community level.

The NPIs collect data on all of the agency’s community activities (not just CSBG-funded activities).

NPI data helps agencies and states analyze what is working and identify steps to improve interventions and impact.

25

What does the current Community NPI data tell us?• Data is disjointed. Doesn’t “add up” to identified

“big” outcomes. No coherent story.

• Number of low income people participating in formal community organizations, etc. is less than the number of CAPs. Seems like underreporting. Suggests NPIs either aren’t understood or aren’t tracked well enough at local level to be reported on.

26

• We are not using population level data such as how many young people enter college, how many adults have some college education, median income, etc.

• NPIs are snapshot measures--have little meaning on their own. Need baseline measures, or measures compared to production (i.e. housing units) or population level data over time to understand what the measures mean. – The NPIs do not provide context. Without an expectation (or

target) of what the agency plans to do, it is not possible to know if they are being successful.

27

• Appears CAPs are good at programs for individuals and families, but when it gets to community-level impact such as increasing the number of living wage jobs in a community, our agencies aren’t there.– Agencies may be contributing to these kind of efforts, but

do not seem to feel they can take credit for this work.

• The numbers reported do not show how CAPs are engaged in larger systemic issues such as those raised at the NASCSP Conference by Sasha Abramsky the author of, “Poverty the American Way”.

28

Community Measures need to….Identify and ‘give credit’ for CAP taking

different approaches to community level work: 1. Independent2. Partnership3. Collective Impact

29

Community Measures need to… Capture the scope or scale of the

impact/intervention. Show movement on larger community or

population level change. Show progress over time. Be easy to understand and allow some

meaningful summary at national level.30

INITIAL FEEDBACK PERIODWHAT YOU TOLD US

31

Responses ReceivedExcellent Response Rate!Survey 2: Community— 731 responsesSurvey 3: Individual/Family— 609 responsesSurvey 4: Fiscal/Agency Capacity— 553

responsesSurvey 5: Tying It All Together— 517 responses

Numerous Letters & Comments also received32

How well does the Community Level Efforts in Progress Report…

33

Table 3: Reactions to Community Level Efforts in Progress Report (Weighted)

How well does the List of Strategies in each domain or section describe all the work you do to ensure communities where people with low-incomes are healthy and offer economic opportunity (Goal 2 &3))?

34

Table 4: Reactions to Community-Level Strategies List (Weighted)

How well does the List of Strategies in each domain or section describe all the work you do to ensure communities where people with low-incomes are healthy and offer economic opportunity (Goal 2 &3))?

35

Table 4: Reactions to Community-Level Strategies List cont. (Weighted)

How well do the Indicators in each domain allow you to capture the impact of the work you do to ensure communities where people with

low-incomes live are healthy and offer economic opportunity?

36

Table 5: Reactions to Community-Level National Performance Indicators (Weighted)

How well do the Indicators in each domain allow you to capture the impact of the work you do to ensure communities where people with

low-incomes live are healthy and offer economic opportunity?

37

Table 5: Reactions to Community-Level National Performance Indicators (Weighted)

Key Issues Identified Community Level Efforts in Progress Report

– The question asking an agency to explain why it does not do community level work is inappropriate.

– Templates need to be streamlined.– Need clearer definitions of independent, partnership, collective

impact.– How are efforts that fold or take a different direction reported?– If we don’t do community work, will we be dropped from the

network? – On the Collective Impact Reporting Tool, it will be very difficult to

accurately measure Behavior/Systems change.38

Key Issues IdentifiedCommunity Level Strategies

– Merge strategies list with Community in Progress tool.– Concrete recommendations for additions to the list.

Community Level NPIs– Some NPIs are not reflective of CAA work; beyond scope of a

CAA.– One CAA alone can’t address the result.– Data for some NPIs is not available.– Baseline requirement is not reasonable.– Fear of small numbers; concerns with some NPIs, showing

correlation between improvements and CAA work.39

CLARIFICATIONS AND CHANGES MADE

40

Critical ClarificationThe TOC driven by our history, knowledge,

experience and the CSBG Act reflects the importance of addressing poverty on both the individual/family and community levels. The NPIs include new robust community level measures related to addressing some of the systemic causes of poverty.

Critical ClarificationIndividual agencies are not expected to

change the wicked and complex causes of poverty alone or overnight. It is critical in today’s world that CAAs

analyze community needs and conditions and aggressively pursue partnerships that are right for that particular agency.

Critical Clarification

Additions to community reporting recognize that: community–level work takes time the local CAA determines and defines

what “community” they are focused on.

Critical Clarification

Agencies only report on those NPIs that are relevant to achieving the goals and outcomes set by the CAA based on local needs and conditions.

Changes Made Report has been extensively modified and simplified so that it

collects basic information on all community level initiatives– Better showcases the connection between initiatives and

outcomes– Demonstrates how initiatives would be reported on over

time– Auto-population where information carries over from year to

year– Demonstrates how outcomes would be tracked and captured

45

Changes Made

Question asking agencies to report why they do not do community level work has been removed.

Elements of Collective Impact (CI) remain, but reporting tool first presented to the Network will be offered as an agency level tool.

46

Changes MadeStrategies list is now integrated into the Community

Level Initiatives Status Page. Captures the number and type of initiatives that

are implemented, and underscores that initiatives are connected to and related to community-level outcomes.

New emphasis on the option to add “Other” in the Initiatives List.

47

Changes MadeCommunity Level Initiatives Forms revised into …..Section A: Community Level Initiatives Community Level Initiatives Home Page

Lists ongoing and completed initiatives, and is the starting place for adding new initiatives

Community Level Initiatives Status Page Data entry form for all community initiative related information This includes Ultimate Expected Outcome, Project Year, Strategies,

information on Partnerships and Collective Impact, and final reporting on Outcomes.

Supplemental Data: Dropdown Lists Initiatives Popup Windows

48

Changes MadeIndicators were revised or reordered to ensure

more accurate representation of community level work.OCS and NASCSP conducted a thorough

review of the social/population indicators included in the proposed Community Level NPIs and made revisions and deletions as necessary.

49

Changes Made• Baseline data is optional based on the

availability of the information for the area being served. Alternately, agencies will have the option to provide a narrative description of why they are working on the initiative, and how they will go about determining a baseline.

50

Changes Made

Now thirty six (36) National Performance Indicators to measure performance of Goal 2.An “Other” Indicator listed in each domain to

allow for locally determined additional outcomes.Community Level Indicators are organized by

six domains. 51

Six Domains1. Employment2. Education and Cognitive Development3. Infrastructure and Asset Building4. Housing5. Health and Social/Behavioral Development6. Civic Engagement and Community

Involvement

52

Employment Community-Level NPI Example

1. The number (and percent) of jobs created to increase opportunities for people with low-incomes available in the specified community.

2. The number (and percent) of jobs maintained for people in the specified community.

3. The number (and percent) of “living-wage” jobs created in the specified community.

4. The number (and percent) of “living wage” jobs maintained in the specified community.

5. Deleted--Decreased rate of unemployment benefit applications. Added--- 5. The number of jobs in the targeted community with a benefit package.

6. Other Outcome Indicator. 53

Employment Initiatives--ExampleMinimum/Living Wage CampaignJob Creation Initiative/Employment Generating ProjectAdded--Job FairsEITC CampaignWIOA One Stop ManagementCommercial Space ManagementEmployer Education on Low-Income IssuesOther: (please specify)

54

Modifications to Module 3-CommunityA Summary

Created the Community Level Initiatives Home Page and Status Page in place of the Community Level Efforts in Progress Report– Significantly streamlined – Integrated strategies– Auto-populate to save time/simplify

Revised the Community Level Performance Indicators– Added flexibility with baseline– Added functionality to reinforce this as a menu– Reduced duplication across NPIs 55

ANNUAL REPORT OUTLINE AND FORMS

56

Module 3: Community Level

Section A: Community Level Initiatives• Community Level Initiatives Home Page• Community Level Initiative Status Page• Supplemental Data: Dropdown Lists• Strategies Popup Windows

Section B: Community Level NPIs• Community Level NPI Landing Page• Community Level NPI Data Entry Forms

57

58

59

60

61

62

63

Strategies Popup64

NPI Landing Page65

Sample NPI Data Entry Form 66

NEXT STEPS

67

Upcoming Webinars

July 6, 2016, 2:00-3:30pm EDT: Module 4 Individual/Family Level

July 14, 2016, 2:00-3:30pm EDT: Module 1 & 2 State Administration & Agency Expenditures, Capacity, and Resources

http://www.acf.hhs.gov/programs/ocs/resource/csbg-dear-colleague-letter-open-comment-period-for-csbg-annual-report

http://www.acf.hhs.gov/programs/ocs/resource/csbg-dear-colleague-letter-open-comment-period-for-csbg-annual-report

70

Resources

www.nascsp.org

Tools and Resources to Support Your Review

Excel Spreadsheets for Modules 2-4State and National ReportsCollective Impact ToolMyth BustersPostersCSBG IS/CSBG Annual Report CrosswalksTools created during the Initial Feedback Period

72

Helpful Links

Link to the Federal Register Notice with submission instructions for comments: https://www.federalregister.gov/articles/2016/06/16/2016-14229/proposed-information-collection-activity-comment-request#t-1

CSBG Dear Colleague Letter Open Comment Period for CSBG Annual Report (with link to Annual Report Packet): http://www.acf.hhs.gov/programs/ocs/resource/csbg-dear-colleague-letter-open-comment-period-for-csbg-annual-report

Information on the OMB Clearance Process: http://nascsp.org/ROMA/919/ROMA-Next-Generation.aspx?iHt=17

Supplemental Tools and Resources to support review: http://nascsp.org/ROMA-Next-Generation/936/FRN1-60-Day-Comment-Period.aspx?iHt=18

THANK YOU!

75