View
4
Download
0
Category
Preview:
Citation preview
FY21 Budget Kick-Off: Program Description Forms – Measures and Targets
JULY 30, 2019
2
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
3
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
4
Where we are going…
Missouri Better Government: meeting Missouri citizens’ expectations so that we are as good as any organization
A common sense, back-to-basics approach to making government better
More transparent… about where we are heading and how we are doing
More citizen friendly… to serve citizens as they deserve
More efficient… to deliver higher quality impact as smart as possible
More focused… on what we should do and scaling back where we should
More accountable… so good public servant performance is recognized
WHERE WE ARE GOING
5
We are focusing on three basic questions to help our teammates succeed
1) Where are we heading?
2) How should our leaders lead?
3) How will we get better?
WHERE WE ARE GOING
6
Effective management in any organization depends upon a basic cycle
Superior and sustained organizational performance
and health
Establishmeasures and clear accountabilities
2
Track performance
4
Create targets, plans, and
budgets
3
Set strategic direction and objectives
1
Decide actions, rewards, andconsequences
6
Hold robust, fact-based performance dialogues
5
Continuous Improvement
WHERE WE ARE GOING
7
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
8
Effective management is driven by a fact-based cycle
Superior and sustained organizational performance
and health
Establishmeasures and clear accountabilities
2
Track performance
4
Create targets, plans, and
budgets
3
Set strategic direction and objectives
1
Decide actions, rewards, andconsequences
6
Hold robust, fact-based performance dialogues
5
Continuous Improvement
WHY MEASURES MATTER
9
“What gets measured gets managed.”
Peter F. Drucker
Why do we care about performance measures and targets?WHY MEASURES MATTER
10
Why good performance measures and targets help us
1) They help us know where we are, what is working, and what is not working as planned (e.g., dashboards, program and project reviews)
2) They help us decide what programs or initiatives we should continue, expand, accelerate, learn from, scale back, stop, or help
3) They help us communicate to our citizens and stakeholders what we do, why we do it, and what we recommend is the right approach (e.g., Program Description Forms; reports; testimony; NDIs)
WHY MEASURES MATTER
11
Our objective with performance measures and targets
Missouri will be among the top tier of all state governments…
… for its strategic and fact-based, approach…
… to decision-making and budgeting…
… that delivers impact for our citizens
WHY MEASURES MATTER
12
Our story so far…
August 2017: Launched effort to improve performance measures and introduce targets for all programs in budget, including first series of capability building and coaching for core staff
October 2017: Departments transmit initial FY19 budget requests to OA Budget & Planning and the General Assembly
October – December 2017: Ongoing problem solving to improve measures and targets for “priority” programs
December 2017: Departments submit Program Description updates for FY19 Budget
January 2018: Governor’s FY19 Budget recommendations submitted
May 2018: COO and OA Budget & Planning introduced new Program Description Form with four categories of measures: activity, quality, impact, and efficiency; launched second series of capability building and coaching for core staff
October 2018: Departments transmit initial FY20 budget requests to OA Budget & Planning and the General Assembly
October 2018: Missouri Budget Explorer website is launched
November 2018: FY20 Budget Program Description Form competition
January 2019: Governor’s FY20 Budget recommendations submitted
March 2019: Missouri Budget Explorer website adds all program measures and targets
July 2019: Third series of capability building and coaching begins for core staff
WHY MEASURES MATTER
13
uses measures to improve transparency and accountability
“Missouri citizens have a right to know how their money is being spent, and featuring department performance measures will further enhance transparency throughout the state.”– State Treasurer Scott Fitzpatrick
“As the chairman of the Missouri Senate’s Appropriations Committee, these performance measures allow lawmakers to take advantage of critical data when it comes to measuring how a state agency or department is using their appropriated resources. This information will provide lawmakers with another tool to hold agencies accountable, and it has the potential to empower lawmakers to be better stewards of state resources.”– Senator Dan Hegeman, Chairman of Senate Appropriations Committee
“As the state budget grows increasingly more complex, I’m glad the administration has put a tool in place to make spending and accountability information readily available to all Missourians.” – Representative Cody Smith, Chairman of House Budget Committee
https://oa.mo.gov/budget-explorer
WHY MEASURES MATTER
14
Our efforts are now being evaluated against national standards of excellence
Source: Results for America, State Standard of Excellence, 2018 Report. https://results4america.org/tools/state-standard-of-excellence-
2018-invest-in-what-works-state-standard-of-excellence/
WHY MEASURES MATTER
15
Real-time interactive survey: How to sign in
Directions:
▪ We will use interactive polling during the workshop
▪ To connect with this capability, please text 22333 the message MOBUDGET
▪ You should receive a text reply “You’ve joined Brittany Ruess’ session (MOBUDGET). When you’re done, reply LEAVE”
▪ You can then send text answers to polling questions during our session today
WHY MEASURES MATTER
16
Real-time interactive poll: How are you feeling now?
Directions:
▪ Text 22333 your answer to this question: “At the start of today’s session, how comfortable do you feel with developing and using performance measures (choose only one)?
▫ Option A: I am comfortable with performance measures and confident I can coach others today
▫ Option B: I am comfortable with performance measures, but I need more practice before coaching others
▫ Option C: I am OK with main concepts, but I need more practice to get comfortable with developing performance measures and coaching others
▫ Option D: I am uncertain about the main concepts related to performance measures and their application, but appreciate their relevance
▫ Option E: I am uncertain with main concepts related to performance measures, their application, and their relevance
▪ If you get disconnected, please text 22333 the message MOBUDGET to reestablish your connection
WHY MEASURES MATTER
17
Real-time interactive poll: What needs more clarification?
Directions:
▪ Text 22333 your answer to this question: “What topic would you like to focus our time on today?”
▫ Option A: Introduction to Program Description Forms
▫ Option B: Development of a central issue statement for a program
▫ Option C: Review of performance measure and target fundamentals
▫ Option D: Introduction to visualizing measures (i.e., the “right” chart)
▫ Option E: Other
▪ If you get disconnected, please text 22333 the message MOBUDGET to reestablish your connection
WHY MEASURES MATTER
18
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
19
Program Description Forms help us improve in two ways
Help us communicate to stakeholders:
Your department and team
Governor’s Office
General Assembly
Our citizens
PROGRAM DESCRIPTION FORMS
Help us manage our programs:
Review program performance
Inform decisions about program and resource allocation
20
Program Description Forms – formatPROGRAM DESCRIPTION FORMS
21
DOLIR, Unemployment Insurance Benefits Program: case study
The Unemployment Insurance (UI) Benefits section’s goal is to promote economic vitality and fairness for Missouri’s businesses and workers by ensuring the UI program is operated effectively, efficiently and in compliance with federal and state laws and regulations
The UI Benefits section accomplishes this by working to ensure timely and accurate Unemployment Insurance benefit payments are made only to those eligible workers who are unemployed through no fault of their own
The benefits allow eligible recipients to focus on returning to work while providing for the basic necessities of life
The purchasing power retained because of this program acts as a stabilizer in times of economic downturn, benefiting Missouri workers, employers and the economy as a whole
Prior to 2017, the program had existing measures provided by the United States Department of Labor that targeted specific program goals
The program also used the USDOL minimum expectation for a successful program for each of the selected measures as targets
Today, the program continues to evaluate the performance measures established by the USDOL The program now uses selected measures that together are indicative of overall program performance Using USDOL standards, they established a base target with stretch targets for each measure
Mission
Impact
Where we are today
Starting point
Activity
Oversight and operations staff at various levels and across program sections were involved in discussion and collaboration to achieve a unified vision for performance measures for the program
Research was done leveraging past and current performance, UDSDOL expectations for performance and comparative data from other state performance to identify Stretch targets for each measure
How we got there
22
DOLIR, Unemployment Insurance Benefits Program: example (1 of 4)PROGRAM DESCRIPTION FORMS
23
DOLIR, Unemployment Insurance Benefits Program: example (2 of 4)PROGRAM DESCRIPTION FORMS
24
DOLIR, Unemployment Insurance Benefits Program: example (3 of 4)PROGRAM DESCRIPTION FORMS
25
DOLIR, Unemployment Insurance Benefits Program: example (4 of 4)PROGRAM DESCRIPTION FORMS
26
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
27
Focus of section
Activity Quality Impact EfficiencyProblem
Is the organization doing what it said it would do?
Is the activity done well?
Does it deliver? Is the activity causing meaningful impact?
Is it worth it? How much effort is invested to achieve the impact?
What specific needor objective is your program addressing?
Tools
Central issue statementIssue trees*
Tool
SMART Measures
A framework for measuring performance and setting targets
*Issue trees are an advanced tool to help identify performance measures. They are covered in future in-depth trainings on problem structuring and performance measures
CENTRAL ISSUE STATEMENT
28
The starting point: What is the problem?
SOURCE: https://www.youtube.com/watch?v=pWgyy_rlmag&t=16s
From the movie “Moneyball” (2011) – viewer discretion advised
WHAT’S THE PROBLEM
29
The takeaway
First, define the problem.
30
A central issue statement provides a starting point for your measuresCENTRAL ISSUE STATEMENT
Basic elements of a program’s central issue statement :
My organization, [fill in: Department, Division/agency], …
is [fill in: key activity such as developing/ coordinating/ overseeing/ monitoring/ implementing, etc.] …
to help [fill in: target audience or whom receives the service or impact – be as specific as possible, e.g., “unemployed Missourians”; “families with loved ones with developmental disabilities”; “Missourians and others interested in enjoying our State Parks”; etc.] …
by [fill in: describe impact on target audience, how you are addressing need, etc.] …
with [fill in: optional – describe additional details about program, if appropriate]
SOURCE: Adapted from the Founder’s Institute
31
This is how a good central issue statement provides the starting point for your Program Description Form
CENTRAL ISSUE STATEMENT
Your central issue statement provides basis for your answer to section 1b on the Program Description Form.
The “is” section of your central issue statement provides the core activities to measure in section 2a.
The “to help” and “by” sections of your central issue statement together provide the impact to measure in section 2c.
32
Illustrative example of a central issue statement: standard program
SOURCE: Department of Corrections team, July 8, 2019 workshop
Illustrative example:
My organization, Department of Corrections, Division of Adult Institutions,…
Is providing Missouri’s correctional facilities with a community standard of healthcare, including medication, hospitalization, psychiatric care, and other specialized care…
To help incarcerated individuals…
By ensuring their quality of care meets constitutional requirements and that they return to their communities healthier than when they entered the institution.
CENTRAL ISSUE STATEMENT
You will likely need to iterate to get to a sharp central issue statement
33
Illustrative example of a central issue statement: grant program
SOURCE: Department of Transportation team, July 10, 2019 workshop
Illustrative example:
My organization, Missouri Department of Transportation,…
Is administering grant funding to AMTRAK …
To help the citizens of Missouri and other travelers…
By ensuring they have reliable, affordable, twice a day rail transportation between St. Louis and Kansas City.
CENTRAL ISSUE STATEMENT
You will likely need to iterate to get to a sharp central issue statement
34
The central issue statement provides the starting point for Budget Program Description Forms
Template
My organization, _________________________________________________________________,
Is ______________________________________________________________________________
________________________________________________________________________________
to help _________________________________________________________________________
by _____________________________________________________________________________
_______________________________________________________________________________
with [optional details on program] ___________________________________________________
_______________________________________________________________________________
Statements should be concise, understandable, and jargon free
CENTRAL ISSUE STATEMENT
35
Group exercise – draft a central issue statement
Template
My organization, Office of Administration, Division of Personnel____________________________,
is __ coordinating a new statewide onboarding process with our sixteen executive departments__
to help _________________________________________________________________________
by __________________________________________________________________________ __
_______________________________________________________________________________
with [optional details on program] ___________________________________________________
_______________________________________________________________________________
Statements should be concise, understandable, and jargon free
CENTRAL ISSUE STATEMENT
36
Illustrative example of a central issue statement – onboarding program
Basic elements of a central issue statement:
My organization, [fill in: Department, Division/agency], …
is [fill in: key activity]…
to help [fill in: target audience] …
by [fill in: describe impact on target audience] …
with [fill in: optional – describe additional details about program if appropriate]
Illustrative example:
My organization, Office of Administration, Division of Personnel,…
Is coordinating a statewide onboarding process with our sixteen executive departments…
To help newly employed team members…
By improving their overall morale, understanding of statewide and department priorities, and thereby their retention…
With programs that leverage best-in-class content and in-person and virtual delivery
CENTRAL ISSUE STATEMENT
37
Checklist to develop a Program Description Form – Part 1
Draft a central issue statement that describes in clear language the program’s: Connection to department strategic goals (What placemat goal does this program connect to?) Core activities (What are your program’s primary activities?) Primary impact (Who is your program helping? How does your program help them?) Additional details on program design, activities, and secondary impacts (if appropriate)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
CENTRAL ISSUE STATEMENT
38
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
39
Activity Quality Impact EfficiencyProblem
Is the organization doing what it said it would do?
Is the activity done well?
Does it deliver? Is the activity causing meaningful impact?
Is it worth it? How much effort is invested to achieve the impact?
What specific needis your program addressing?
Tools
Central issue statementIssue trees*
Tool
SMART Measures
A framework for measuring performance and setting targets
*Issue trees are an advanced tool to help identify performance measures. They are covered in future in-depth trainings on problem structuring and performance measures
HOW TO MEASURE
Focus of section
40
Good measures are the foundation for fact-based performance management
Key points to remember▪ It is better to start with a “good enough” measure rather than wait for
perfection▪ If data for a good measure is currently not being collected, start with what you
have while putting in place data collection for the new measure (e.g., a new citizen survey)
Purpose of measures
▪ Provide a quantifiable or sometimes qualitative metric that reflects how well an organization is achieving its stated goals and objectives
▪ Clarify a program’s objective or what it is really aiming to accomplish
▪ Help leaders, staff, and stakeholders focus on what matters most strategically and what they need to accomplish for the year
▪ Provide transparency for decision making and public communication
▪ Establish a fact-based foundation for performance management – including resource allocation decision making
HOW TO MEASURE
41
Good measures follow the “SMART” principle
▪ Does it have a clear definition?▪ Is it straightforward and easy to understand?
Simple
▪ Is it easy to measure?▪ Can it be benchmarked against other organizations or outside data?Measurable
▪ Do we understand the drivers that are behind the measure?▪ Can we take action to deliver impact?Achievable
▪ Is the measure aligned with the State’s and/or the department’s strategy and objectives?Relevant
▪ Is the time to impact defined and is it practical?▪ When will we monitor it? Can the measure move between
periods?
Timely
HOW TO MEASURE
42
Activity Quality Impact EfficiencyProblem
Are your children doing the work?
How many seeds are they planting per day in the first month?
How many seeds are they watering per day during the season?
Are your children doing the work well?
How many seeds germinate during the first month?
What is the ratio of productive to non-productive plants during the second month?
Is the garden providing vegetables and spending money?
What is the amount of profit earned by selling vegetables during the season?
Your children need extra spending money.
Their plan is to grow a vegetable garden.
A framework for measuring performance – exampleHOW TO MEASURE
Was the effort worth it?
How many $/hour of work did they make compared to their alternative sources of income during the season?
43
Activity Quality Impact EfficiencyProblem
Departments want new team members who understand department priorities, team priorities, and how they contribute to the organization.
Your department plans to establish a standard onboarding process to achieve these goals.
Is the organization doing what it said it would do?
Is the activity done well?
Does it deliver? Is the activity causing meaningful impact?
Is it worth it? How much effort is invested to achieve the impact?
HOW TO MEASURE
Group exercise – a framework for measuring performance
44
Targets show whether a program is on track or not
Key points to remember▪ Targets should be practical and specific▪ Targets should roll up so that they would “move the needle” on a strategic
priority▪ Try to set individual program targets at two levels – base (minimum
acceptable) and stretch (truly distinctive)
Purpose of targets
▪ Set clear and transparent expectations for performance
▪ Get everyone on the same page through agreement on what constitutes good and bad performance
▪ Prevent surprises or consequences
▪ Help leaders, staff, and stakeholders focus on exactly what matters most –i.e. results – and what they need to accomplish for the year
▪ Clarify accountabilities or ownership, which increases the probability of delivery and improves organizational alignment
HOW TO MEASURE
45
Performance targets can be set using a combination of approaches
Approach Description When to use
Continuous improvement (top-down)
▪ Consistent year-to-year incremental (e.g., 3%-5%) performance improvement targets
▪ Organization is already high performing
Grass roots(bottom-up)
▪ Managers at every level develop improvement plans and define achievable targets, then iterate to align plans with top-down need
▪ Organization depends on frontline buy-in, motivation, and ownership
Transformational(top-down)
▪ Step-change targets that require fundamentally redesigning the way work is done
▪ Organization must improve its performance through a step-change to survive or preserve support (e.g., program on verge of being discontinued)
Theoretical limit(top-down)
▪ Absolute limits used to set targets (e.g., technical limits, theoretical limits based on laws of nature)
▪ Organization possesses motivation and resourcefulness to drive toward the most challenging targets
External benchmark
▪ Other organizations’ performance under similar operating conditions is used to develop aspirational targets
▪ Organization is a below average performer with aspirational goal of becoming top tier, best-in-class
▪ Comparable organizations exist
Internal benchmark
▪ Best period performance for comparable units under similar operating conditions is used to develop challenging targets
▪ Units have similar characteristics and can be compared
▪ Organization tracks performance and can share information across units
Most common
HOW TO MEASURE
46
Tip: start by looking at how others measure performance and set targets
Washington: http://www.results.wa.gov/ Maryland: https://dbm.maryland.gov/
Tennessee: https://www.tn.gov/transparenttn.htmlMinnesota: https://mn.gov/mmb/mn-dashboard/
Ask “who is the best in the country at this sort of program or activity? If a direct one-to-one comparison does not exist, what other organizations might provide analogies?”
Research other states, professional associations (e.g., National Association of State Chief Administrators ), federal government, and even private sector organizations measure
HOW TO MEASURE
47
Tip: Monitoring performance measures through time helps you assess programs and adjust course if needed
Activity
≠
Quality Impact
If the “right” activities…
Are done consistently with good quality…
But the impact does not improve…
Then it is time to rethink the program design
HOW TO MEASURE
48
Checklist to develop a Program Description Form – Part 2
Draft a central issue statement that describes in clear language the program’s: Connection to department strategic goals (What placemat goal does this program connect to?) Core activities (What are your program’s primary activities?) Primary impact (Who is your program helping? How does your program help them?) Additional details on program design, activities, and secondary impacts (if appropriate)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
HOW TO MEASURE
Develop version 1.0 performance measures and targets for your program: Research “who is the best in the country at this sort of program or activity? If a direct one-to-one
comparison does not exist, what other organizations might provide analogies?” Review how other organizations measure performance and define their targets: where
appropriate, adapt; where needed, improve or create new measures and targets Draft version 1.0 measures and targets
Core activities: start with “what is the most important activity to implement the program?” Quality: then “how to we measure the quality of the core activities?” Impact: consider “what impact matters most to our citizens and/or major stakeholders?” Efficiency: this typically involves some sort of ratio or relationship between activities (or
total effort) and impact
Review measures and targets to ensure they meet best practice standards: You should be able to describe all the measures and targets in clear language All measures use SMART principles All targets are clearly identified
49
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
50
Exercise: How to apply these approaches to your program?
Objective: Apply the basic principles introduced today to an example from your department
Go to your break-out room in your department teams
Identify a scribe for your team to take notes on the flip chart
Identify a program from your department to use for this exercise Ideally, you will have identified this program beforehand; if not, spend only a few minutes
selecting a program for the sake of this exercise Note: if you want to divide into more than one team so each can work on its own program
description, feel free to divide up
Complete the following steps: Step 1: Record your program title at the top of the flip chart Step 2: Draft a central issue statement using the template format [~15-20 minutes] Step 3: Develop at least one measure for each category (activity, quality, impact, efficiency)
[~25-30 minutes]
Time permitting, select a target for one measure and provide your reasoning for why this is the appropriate target
Return to plenary and be prepared to share highlights of your problem solving and any specific challenges or questions you have related to your program
APPLICATION
51
Template – central issue statement
Template
My organization, _________________________________________________________________,
Is ______________________________________________________________________________
________________________________________________________________________________
to help _________________________________________________________________________
by __________________________________________________________________________ __
_______________________________________________________________________________
with ___________________________________________________________________________
_______________________________________________________________________________
Statements should be concise, understandable, and jargon free
APPLICATION
52
Activity Quality Impact EfficiencyProblem
Draft core issue statement
Is the organization doing what it said it would do?
Is the activity done well?
Does it deliver? Is the activity causing meaningful impact?
Is it worth it? How much effort is invested to achieve the impact?
APPLICATION
Template – draft performance measures and targets
53
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
54
Getting your presentation of your measures and targets right matters
SOURCE: Infogram. https://infogram.com/page/data-visualization
Getting your presentation of data right makes it easier for people to understand your measures and targets
The best data “visualizations” are clear, precise, and efficient
Quality data visualizations will:
Make your measures easier to understand and remember
Demonstrate relationships and patterns quickly
Help show and explain trends and any outliers
Provide insight that allows us to make better decisions
DATA VISUALIZATION
55
Avoid these common mistakes with data visualizationDATA VISUALIZATION
SOURCE: https://www.youtube.com/watch?v=MjcO2ExtHso&t=146s
From Don McMillan “Life after Death by PowerPoint” (2012)
56
Choosing the right chartWhat comparison are you making?
Component
Ranking
Time series
Frequency distribution
The size of each part as a percentage of the whole
The relative rank of a group of items: are they the same, or is one more or less than the others?
The trend in how items change over time
How many items fall into a series of ranges
“Share” “Percentage of total” “Accounted for X percent”
“Larger than” “Smaller than” “Equal”
“Change” “Growth” “Decline”
“Concentration” “X to Y range” “Frequency”
Description Commonly used words
Source: Say It With Charts: The Executive’s Guide to Visual Communication, by Gene Zelazny
DATA VISUALIZATION
57
Types of charts
What type of comparison are you making?
Ranking
Component
Distribution
Between groups Bar chart
Line chart
Static
Over time
Pie chart
Stacked bar chart
One variable Histogram
Time series
Comparison type Standard chart
DATA VISUALIZATION
58
Component comparisons (example)
FY18 IT department expenditures as a share of total IT spend
30%
6%
6%
5%4%3%
17%
4%
11%
1%
3%
DSS
0%0%
DHE
MoDOT
DSS MMIS
DIFP
DOR
3% ITSD3%
MDC
DOLIR
DHSSDMH
1%MDA
DPSDED
DNRDOCOA
DESE
1%1%
The size of each part as a percentage of the whole
DATA VISUALIZATION
SOURCE: OA ITSD
59
Ranking comparisons (example)
% of State of Missouri staff turnover by department, January 1, 2018 to December 31, 2018
The relative rank of a group of items: are they the same, or is one more or less than the others?
SOURCE: OA Personnel – LinkedIn Learning data
DATA VISUALIZATION
60
Time series comparisons (example)
% change since 2013 in job postings vs. job applicants, 2013-2017
The trend in how items change over time
0
86 6
11
0
-9
-20
-24 -24
-25
-20
-15
-10
-5
0
5
10
15
2013 2014 2015 2016 2017
35%
Job postings
Job applicants
SOURCE: OA Personnel – LinkedIn Learning data
DATA VISUALIZATION
61
Distribution comparisons (example)
Distribution of March 2019 Quarterly Pulse Survey responses throughout survey period
How many items fall into a series of ranges
3,6003,400
1,500
1,2001,070
2,800
1,500 1,500
2,500
2,2002,300
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
15-Mar14-Mar13-Mar 18-Mar 19-Mar 20-Mar 21-Mar 22-Mar 25-Mar 26-Mar 27-Mar
SOURCE: Quarterly pulse Survey, March 2019.
DATA VISUALIZATION
62
A simple check to help guide you
If a reader cannot interpret the overall story you are telling within
10 seconds, it’s time to simplify
SOURCE: LinkedIn Learning, “Excel Data Visualization Part 1” with Chris Dutton.
DATA VISUALIZATION
63
Put the check to the test….
A quick group exercise…
Raise your hand when you understand what the story that the chart is saying below
SOURCE: LinkedIn Learning, “Excel Data Visualization Part 1” with Chris Dutton.
DATA VISUALIZATION
64
Put the check to the test….
A quick group exercise…
Raise your hand when you understand what the story that the chart is saying below
DATA VISUALIZATION
65
Group exercise: practice selecting charts
Objective: Practice identifying which chart to use to answer your questions using data
On the following page, take 5 minutes to sketch out up to two charts using the data provided below to best answer the question:
Is Department X meeting its goal of retaining employees?
Consider: What story are you trying to tell? What kind of comparison are you trying to make between the data points?
Identify one type of additional data that you would go collect to best answer this question
Partner up with someone sitting next to you to discuss the following (5 minutes): What story are you telling? Which charts did you choose and why? What additional data would you like to have?
Target Jun-18 Jul-18 Aug-18 Sep-18 Oct-18 Nov-18 Dec-18 Jan-19 Feb-19 Mar-19 Apr-19 May-19
1% 1.77% 2.20% 1.34% 1.96% 2.00% 1.96% 2.00% 1.60% 2.06% 1.62% 1.98% 1.74%
Employee turnover rate by month
DATA VISUALIZATION
66
Group exercise: practice selecting charts
Chart 1
Chart 2
DATA VISUALIZATION
67
Checklist to develop a Program Description Form – Part 3
Draft a central issue statement that describes in clear language the program’s: Connection to department strategic goals (What placemat goal does this program connect to?) Core activities (What are your program’s primary activities?) Primary impact (Who is your program helping? How does your program help them?) Additional details on program design, activities, and secondary impacts (if appropriate)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
Develop version 1.0 performance measures and targets for your program: Research “who is the best in the country at this sort of program or activity? If a direct one-to-one
comparison does not exist, what other organizations might provide analogies?” Review how other organizations measure performance and define their targets: where
appropriate, adapt; where needed, improve or create new measures and targets Draft version 1.0 measures and targets
Core activities: start with “what is the most important activity to implement the program?” Quality: then “how to we measure the quality of the core activities?” Impact: consider “what impact matters most to our citizens and/or major stakeholders?” Efficiency: this typically involves some sort of ratio or relationship between activities (or
total effort) and impact
Review measures and targets to ensure they meet best practice standards: You should be able to describe all the measures and targets in clear language All measures use SMART principles All targets are clearly identified All charts adhere to the 10 second rule and follow best practices for displaying information (e.g.,
component, ranking, time series, distribution)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
DATA VISUALIZATION
68
Today’s agenda
Time Topic
1:00-1:10 pm Where we are going
1:10-1:25 pm Why measuring performance matters
1:25-1:45 pm Program Description Forms – overview and case study
1:45-2:15 pm What’s the problem – central issue statement
2:15-2:35 pm How to measure – developing SMART measures and targets
2:35-3:30 pm How to apply these tools – exercise with department examples
3:30-3:50 pm How to visualize data
3:50-4:00 pm Conclusion
69
Real-time interactive poll: How are you feeling now?
Directions:
▪ Text 22333 your answer to this question: “At the end of today’s session, how comfortable do you feel with the content (choose only one)?
▫ Option A: I am comfortable with the content and confident I can coach others today
▫ Option B: I am comfortable with the content, but I need more practice before coaching others
▫ Option C: I am OK with main concepts, but I need more practice to get comfortable with their application
▫ Option D: I am uncertain about the main concepts and their application, but appreciate their relevance
▫ Option E: I am uncertain with main concepts, their application, and their relevance
▪ If you get disconnected, please text 22333 the message MOBUDGET to reestablish your connection
CONCLUSION
70
Critical roles to get work done by October 1
Department-level and division-level leadership Reinforce importance and expectations of quality measures Participate in final review and refinement of Program Description
Forms/measures before submission
Department program leaders Build capabilities within teams to develop and use performance measures Research other states’ examples of measures, targets, etc. Review and update all Program Description Forms with focus on identifying
priorities, clarifying program design, and improving measures and targets
Department-level budget and Operational Excellence teams Build capabilities within teams to develop and use performance measures Provide coaching and problem solving help to department program leaders and
teams Coordinate internal department reviews, quality assurance, and transmission of
documents to OA, Budget & Planning
Governor’s Office team and OA Budget & Planning team Support Governor ’s Office review Develop and implement training, feedback, and additional work sessions Provide coaching and problem solving support
CONCLUSION
71
What’s next? FY21 budget planning calendar
July – September 2019: State agencies meet internally to analyze their budgets and use performance measures to consider possible changes for FY21 budget including:
Reallocating existing resources and staff Reducing existing appropriation levels Requesting additional appropriations for mission based programs
August 2019 (date TBD): Office of Administration, Division of Budget and Planning and Operational Excellence hold “office hours” coaching sessions to help department teams with specific performance measure and target challenges
August 2019 (date TBD): Chief Operating Officer with OA, Operational Excellence will hold additional capability building sessions on performance measures and targets
October 1, 2019 - DEADLINE: Departments submit FY21 operating budget recommendations to Governor Parson via the OA, Division of Budget and Planning
October – December 2019: Budget and Planning works with the departments to analyze the FY21 budget requests for Governor Parson’s review and recommendations
Departments review and revise as needed top priority Program Description Forms Second annual Program Description Form competition to recognize the best Program Description Forms
and data-based presentations in State of Missouri
Mid-January, 2020: Governor Parson presents his FY21 budget recommendations to the citizens of Missouri
CONCLUSION
72
Activity Quality Impact EfficiencyProblem
Is the organization doing what it said it would do?
Is the activity done well?
Does it deliver? Is the activity causing meaningful impact?
Is it worth it? How much effort is invested to achieve the impact?
What specific needis your program addressing?
Tools
Central issue statementIssue trees*
Tool
SMART Measures
A framework for measuring performance and setting targets
*Issue trees are an advanced tool to help identify performance measures. They are covered in future in-depth trainings on problem structuring and performance measures
CONCLUSION
73
Checklist to develop a Program Description Form
Draft a central issue statement that describes in clear language the program’s: Connection to department strategic goals (What placemat goal does this program connect to?) Core activities (What are your program’s primary activities?) Primary impact (Who is your program helping? How does your program help them?) Additional details on program design, activities, and secondary impacts (if appropriate)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
Develop version 1.0 performance measures and targets for your program: Research “who is the best in the country at this sort of program or activity? If a direct one-to-one
comparison does not exist, what other organizations might provide analogies?” Review how other organizations measure performance and define their targets: where
appropriate, adapt; where needed, improve or create new measures and targets Draft version 1.0 measures and targets
Core activities: start with “what is the most important activity to implement the program?” Quality: then “how to we measure the quality of the core activities?” Impact: consider “what impact matters most to our citizens and/or major stakeholders?” Efficiency: this typically involves some sort of ratio or relationship between activities (or
total effort) and impact
Review measures and targets to ensure they meet best practice standards: You should be able to describe all the measures and targets in clear language All measures use SMART principles All targets are clearly identified A charts adhere to the 10 second rule and follow best practices for displaying information (e.g.,
component, ranking, time series, distribution)
Ask someone outside your program to review the draft central issue statement to ensure clarity, and revise as needed
CONCLUSION
74
Additional notes
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Recommended