WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 1
What Works Cities Certification:What Excellence Looks like in Local Government
By Stephen Goldsmith
WHAT WORKS CITIES March 2017 | 2WHAT WORKS CITIES 2
Introduction
What Works Cities, an initiative of Bloomberg Philanthropies,
pairs mid-sized cities with expert partners – the Behavioral
Insights Team (BIT), the Center for Government Excellence
(GovEx) at Johns Hopkins University, the Government
Performance Lab (GPL) at the Harvard Kennedy School,
Results for America and the Sunlight Foundation – for
technical assistance in better using data and evidence.
After two years of work with 77 cities across the United
States, the appropriately named program identified the
key characteristics of a city devoted to using data to
comprehensively identify, well, what works. Driven by the
desire to share that knowledge more broadly, What Works
Cities designed its newly announced Certification program
to recognize high-performing cities across those criteria, to
create an objective standard of success, and to help cities
at any point in the data journey understand how they can
improve their practices. As Jenn Park, Associate Director
for What Works Cities, said, “We want to be able to show
the world what the best cities are doing. The Certification
program is made to be able to do just that – publicly
validate, recognize, and celebrate cities that are doing this
at the highest level.”
Certification measures a city’s work across criteria in the
domains of open data, data governance, performance
analytics, low-cost evaluations, results-driven contracting,
and repurposing for results. The What Works Cities
Standard – commit, measure, take stock, and act – has
guided the What Works Cities initiative from the beginning,
and the Certification criteria are divided into those four
areas. The Standard represents phases of a city’s work
to use data and evidence effectively, beginning with a
mayor’s public commitment and concluding with using
a deep understanding of city data to inform major policy
and program decisions. Simone Brody, Executive Director
of What Works Cities, described the Standard as “the
North Star of what this work should look like.” She noted
that, based on demand from cities for a tactical guide
to improving practices, Certification takes the theoretical
Standard and translates it into concrete indicators.
COMMIT MEASURE TAKE STOCK ACT
What Works Cities leaders make
powerful, public commitments to
getting better results for their residents by
using data and evidence.
What Works Cities leaders use data and
tools at their disposal to measure progress and engage residents
along the way.
What Works Cities leaders consistently
review and reflect to measure progress, learn, and makecorrections and
improvements.
What Works Cities leaders use data and
evidence to inform major decisions and
take action.
The What Works Cities Standard
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 3
Any city with a population over 30,000 is eligible to apply
for Certification, and after a robust evaluation of their
efforts, high-achieving cities will be recognized with silver,
gold, or platinum certification. Applicants will be able
to benchmark themselves against their peers and get a
clear sense of where their individual practices are and in
what areas they can move forward. Although What Works
Cities’ technical assistance is limited to mid-sized cities,
participating in Certification allows far more cities to access
resources and participate in a growing community of cities.
What Works Cities Certification fits into the existing
landscape of initiatives in this space by recognizing
governments that have developed a broad, citywide
capacity for using data and evidence, rather than
awarding specific successful initiatives. By measuring
aspects such as establishing a person or team responsible
for data standards and protocols, developing a process for
releasing open data, and measuring the outcomes of key
procurements, Certification focuses on the fundamentals
of data-driven government in a way that other recognition
programs do not. Elevating the day-to-day city work and
processes that results in dramatic successes is an important
contribution to the field.
Brody emphasized that even a city just starting out can
benefit from the process, noting that the keys to success are
accessible to any city: “What we’ve found is most important
to being effective at this is a real commitment from senior
leaders in cities, a real belief this is going to improve
outcomes, and then giving folks in city government the
space to be innovative and try new tools and practices. Any
city can do that if they want to, and we’ve seen dramatic
progress in cities just starting out, even in a few months.”
Case Studies of Platinum Practices
What Works Cities Certification evaluates applicant
cities on 50 criteria within open data, data governance,
performance analytics, low-cost evaluations, results-driven
contracting, and repurposing for results. The criteria are
focused on the people, programs, and policies necessary
to improve the effectiveness of government. Below, we
highlight examples of the cities in the What Works Cities
community that are already achieving selected criteria
at the highest levels, both to provide inspiration and to
illustrate what a top example of each criterion looks like.
Commit: Has your local government defined and made publicly available time-bound, measurable local government-wide strategic goals (e.g., reduce homicide by 20% in three years)?
The City of South Bend, Indiana’s notable ability to set
strategic goals has improved city government in a major
way, helping Mayor Pete Buttigieg deliver on critical
priorities and driving structural changes in the way the
city addresses problems and services. By setting clear
goals that drive work throughout the city, and reporting
on those goals to residents, Buttigieg has created a high-
performing government that is accountable for results. One
outstanding example is the publicly stated strategic goal
of addressing 1,000 vacant or abandoned properties in
1,000 days, which started in early 2013. Mayor Buttigieg
wanted to tackle the issue of blight, which residents told
him was a priority during his campaign, in a visible way
that allowed the community to track the city’s progress.
The city’s commitment to addressing the vacant properties
was measurable and available on the city’s website. Even
when the process had issues, the public value was clear.
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 4
Commit: Does your local government have a codified open data policy?
The City of Seattle, Washington, was one of the first U.S.
cities to pursue open data, creating the first iteration of its
open data portal in 2010 under Mayor Michael McGinn,
a prominent proponent of government transparency. Since
then, Seattle has established itself as a leader in the field,
consistently increasing the volume and accessibility of
available information. In 2015, Seattle was named a What
Works city, and according to Seattle’s Chief Technology
Officer (CTO) Michael Mattmiller, it came at the perfect
time for building the city’s open data capabilities. “We
were already thinking about how to re-engage the city and
expand the open data portal’s use,” Mattmiller said. “One
of the areas that we identified to focus on was this notion
of creating an open data policy.” The city began work
with What Works Cities partner the Sunlight Foundation to
develop this policy. “It was very helpful for us to have the
model policy language, to have specific policy objectives
that we could work towards,” said Mattmiller. “But, we also
realized that we had some unique aspects of Seattle that
we had to mediate.”
For Seattle, it was important to develop a policy that fit
the needs of the community, particularly in the realm of
privacy. Mattmiller explained, “Before the What Works
Cities engagement started, we had several missteps in our
community about how we collected and used residents’
data. … When we thought about opening more datasets,
we felt the tension that our community was going to
have between seeing this as a win for transparency and
economic development and concern about what that
data might do in terms of causing privacy harms.” The
partnership with What Works Cities provided an impetus to
mold an open data policy that worked for Seattle residents.
The city partnered with the University of Washington and
Local media picked up on a bug in the progress-tracking
system that erroneously showed 100 pending properties
as already addressed. The city’s Chief Innovation Officer
Santiago Garces said this media revelation led to structural
changes in the way that the city was tracking its progress
with code enforcement. These changes – which included
simplifying inspector checklists, requiring inspectors to
take pictures of the properties, and assigning a central
data analyst to do quality assurance – allowed the city
to “improve the speed at which we were addressing the
properties, and we actually exceeded the goal that we had
set,” Garces said.
Other strategic goals laid out by the South Bend city
government include ensuring transparency and equity
in policing, enhancing physical and technological
infrastructure, and addressing mobility. In addition to
addressing public concerns and creating action-driven
strategic goals, the city consulted with the Drucker Institute
as well as the Center for Priority Based Budgeting, What
Works Cities, and GovEx to help with the framing of those
goals. Garces said working with outside groups was critical
to building the city’s “operational capacity and framework,”
and making tangible goals that address public concerns
has been critical in building trust with residents.
A key thread running through the strategic goals is the
emphasis on reporting progress and critical information
to the public. The city is working to create transparency-
oriented microsites on its open data portal that will report
data and contextual information about specific goals to the
public. The first such site, which is set to be released in the
spring of 2017, will focus on the strategic goal of “making
sure the city has a 21st-century police department.”
Garces added that these microsites will help the city better
tell the story of what the city is trying to achieve and how it
is progressing toward its goals.
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 5
received a grant to develop a municipal privacy program,
creating an action committee to establish a set of privacy
principles to include in the open data policy. The city then
reached out to the Seattle community for feedback on the
policy, making a number of changes to the types of data
to be opened. Seattle also established a network of open
data champions in the city government to examine data
before publication to monitor for invasive personal info
and potential mosaic effects – combinations of datasets
that, together, provide private information. To implement its
privacy and open data policies, Seattle has partnered with
the Future of Privacy Forum to identify and help mitigate
risk present in its open data program, and contributed to
research led by the Berkman-Klein Center at Harvard to
share its policies with other cities.
However, publishing and protecting data was only the
beginning for Seattle; the city then had to encourage
city employees and residents to use that data. Seattle
began convening the open data champions and other
city employees each month in an “Open Data Breakfast
of Champions,” bringing in guest speakers to talk about
applications of data. “Through these meetings, we keep
people enthusiastic and give them best practices,” said
Mattmiller. Seattle also held a Data Camp, in which the
city took employees off the job for three days for training
on how to use the open data portal, in addition to other
data skills. Moreover, to promote resident engagement
with data, the city created the position of Civic Technology
Advocate, a data leader that goes into communities and
hosts meetings, hackathons, and design labs to spread the
goals of the open data program and empower residents
to use municipal data. As a result, the city has seen the
development of a number of useful tools and applications.
For example, thanks to a Park Hackathon, developers
created a tool using Parks Department trail data that helps
users navigate Seattle’s parks.
By creating an open data policy that engaged users and
fit the needs of Seattle employees and residents, the city
was able to invigorate and institutionalize its open data
program.
Measure: Does your local government maintain a comprehensive data inventory?
Kansas City, Missouri’s comprehensive data inventory
shows the importance of clear internal structures and
processes to maintaining a successful, sustainable open
data program. When Chief Data Officer Eric Roche
realized how much time he was spending updating out-
of-date, non-automated open data in the city’s portal,
he embarked on a project to understand and inventory
the data in all departments to develop a more systematic
approach to publishing open data in the city.
Roche took a methodical approach to building the
inventory: he drew on the relationships he had built
through the city’s performance management program,
asked for organizational charts, and talked to individual
departmental representatives. Through What Works Cities,
the Sunlight Foundation and GovEx provided guidance
on the inventory process. Roche acknowledged that not
all department officials were data systems experts, but
nonetheless, he and his team were able to find the answers
they needed: “We asked what kind of work the departments
do, how they track that work, where they store it, and then
backed our way into the more technical questions.”
This process has proved fruitful; Roche has been able
to identify people who “speak data” in several city
departments, and this has led to positive relationships
that yield results beyond the inventory. This peer-to-peer
work in the city government has been the key ingredient
in building a comprehensive data inventory for Kansas
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 6
City. Roche said that the biggest lesson he learned was to
“start small” – the city originally planned to complete an
inventory of seven departments in 60 days, but adapted
the plan to incorporate departments in an ongoing way
that also builds capacity for data in other city departments.
Roche said the development of a citywide data inventory
has given the city a thorough, well-documented resource
that facilitates a more effective open data strategy. It allows
the city to prioritize data releases based on key priorities
and what can be automated, instead of just the “low-
hanging fruit.” Technical difficulties are a common barrier
to publishing certain city datasets, Roche said, as data
systems are not always compatible with publishing online,
but the inventory has served as a critical resource for the
city in navigating such challenges. “The inventory gives me
the ability to move on to the next thing,” Roche said. “It
gives me the sense that there’s more out there – there’s a
lot more valuable data to grab at any given moment.”
Measure: Does your local government measure outcomes, impacts, and/or cost-effectiveness of key procurements, contracts, and/or grants (i.e., monitor performance data in real-time and troubleshoot with contractors to achieve the goals of the contract and/or grant)?
Boston, Massachusetts, has robust open data, performance,
and analytics programs, so when the opportunity to
engage with What Works Cities experts arose, government
leaders looked to apply the power of data to their contracts
through results-driven strategies. With the help of Elijah de
la Campa, a Fellow from GPL, the city focused its efforts on
the Department of Public Works’ Construction Management
Division, which manages numerous contracts each year.
Each year, Boston spends nearly $8 million on an asphalt
resurfacing program for its 800 miles of streets. To ensure
an equitable distribution of repairs, the city divides this work
into three geographical regions and accepts bids for each.
While the prior contracts included technical standards
related to the quality of asphalt resurfacing, there were few
mechanisms in place to enforce or incentivize vendors to
adhere to the standards. Modifying the contracts for this
program offered the city a chance to increase the overall
quality of repaving efforts, to improve communication and
transparency with vendors, and to enhance the articulation
and measurement of outcomes crucial to the asphalt
resurfacing process.
The Department of Public Works and GPL began assessing
the existing procurement process by gathering information
from stakeholders. De la Campa emphasized the
qualitative and human-centric nature of this work as he
spent considerable time meeting with city engineers and
vendors to understand concerns with the program, how
they could be best addressed, and the viability of different
types of performance payment.
In its new asphalt resurfacing contracts, the city has defined
outcomes of interest related to pavement quality, the speed
and progression of paving operations throughout the city,
parking management, and environmental management,
among others. Because the data to rigorously measure
these aspects of performance did not yet exist, the city set up
new processes for its engineers to track data. The contracts
are now written with a clear set of outcome metrics, which
are incentivized with a new performance-based payment
structure. In addition to offering performance payments for
meeting pre-specified progression of work benchmarks,
the city will grade each contractor’s performance three
quarters of the way through the paving season, and then
award additional in-season work for the final quarter
according to vendor performance. The vendors benefit
from the clear information about the city’s expectations
and the incentives for high-quality performance.
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 7
Boston has now hired its pavers for 2017 using the new
contracts and will implement the performance-based
payment structure for the first time this paving season.
The new approach of results-driven contracting has many
more applications throughout the city’s operations to help
Boston deliver better services to its residents.
Take Stock: Does your local government convene a performance management program (i.e. Stat meetings)?
The City of Louisville, Kentucky’s performance management
system, LouieStat, sets the bar for city government
performance improvement. Mayor Greg Fischer united
lessons from his business background with existing
government stat models and unveiled LouieStat in 2012 to
focus on two areas: planning and operations. “We needed
to figure out how to plan, and we created consistent
guidelines and language and a single coordinated strategic
planning process that would help us measure the strategic
areas of focus,” said Daro Mott, Chief of Performance
Improvement in Louisville. “We also needed something
that was more operational, which would have us measure
the critical business processes – the processes that deliver
the core of citizen services. We really needed to create a
program that could answer the question of how Louisville
could continuously improve on service delivery.”
Mott said that breaking the work into distinct strategic and
operational categories was critical for the success of the
system. “Operations should flow from the strategy of the
city. … If you start with data that you already have, you
may not develop the right performance measures. You
need to ask, ‘What are we planning to do, and what data
will help us understand how well we’re doing the work?’”
This way, a city’s performance management efforts will
center around its strategic priorities, rather than boosting
performance on arbitrary metrics.
As a part of the planning process, Mayor Fischer developed
a six-year plan with 21 city goals and asked each agency
to develop its own goals and plans to achieve them. The
Mayor’s senior leadership meets with senior staff from
18 of 20 departments four times a year and with other
staff members between these forums. In these meetings,
attendees discuss progress, look at metrics for the
department and identify areas of weakness, evaluate the
impact of city programs, and make data-driven decisions
about where and how to best allocate resources. The
Mayor attends many of these forums himself and also
meets with Mott on a regular basis to analyze Louisville’s
performance on a citywide level. Mayor Fischer said what
he calls a “weakness orientation” is key to making these
meetings productive instead of punitive: “Bad stat programs
are human- and people-focused and create more of a
blaming culture. Ours is a celebration culture, focused on
identifying broken processes or bad data and then fixing
that and celebrating the people who do the work.”
In order to promote buy-in from so many departments,
LouieStat, from the beginning, sought to demonstrate its
utility to agencies. According to Mott, “What really got us
more buy-in was facilitating process discovery workshops
with departments, by which we documented the critical
business processes of each department and talked about
measures linked to these processes.” In doing so, the
Mayor’s Office introduced departments to performance
management – and showed how performance management
could help identify and track metrics to improve service
delivery.
The performance management culture has become
increasingly embedded in Louisville’s agencies. Mayor
Fischer points to this as a critical aspect of developing a
culture of performance; he said, “We provided training
for people to understand how to solve problems, which
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 8
has given them a sense of not just empowerment, but
fulfillment and hopefully joy in their work, where now they
feel they are in control of making things better.” The Office
of Performance Improvement has trained at least one staff
member in each agency to lead the LouieStat process
and analyze that department’s data. Most data analysis
now happens at the departmental level, and agencies
have come to embrace a performance-based approach,
learning to adapt LouieStat to their various needs.
Act: In the last 12 months, has your local government used the results from low-cost or randomized evaluations to improve local government programs or processes?
In the past year, Washington, D.C., has dramatically
increased its efforts to use low-cost evaluations in
policymaking. Boosted by commitments to data-driven
decision-making from the Mayor and City Administrator,
the city launched The Lab @ DC last year. The Lab brings
diverse scientific skill sets in house to enable the city to
use low-cost interventions and other research methods
throughout its operations.
By basing the team in the Office of the City Administrator,
The Lab builds on existing relationships, processes, and
data infrastructure. Chief Performance Officer Jenny Reed
noted that the connection to performance management
surfaces ideas and also ensures that the work is tied to
the city’s priorities. Lab Director David Yokum said that,
in order to identify opportunities for low-cost evaluations,
“Having scientists inside government is a strength. You
really need to know a lot about the agencies, what they
are capable of doing, what their budgetary constraints are,
what their IT looks like – you need all those pieces to make
the scientific judgment of what the opportunities are.”
The Lab is already embarking on a variety of efforts,
including testing redesigned paperwork for applications
to Temporary Assistance for Needy Families (TANF). For
a project with the Metropolitan Police Department (MPD),
The Lab designed a randomized controlled trial for the
rollout of body-worn cameras. Because the MPD was
already planning to distribute the cameras and already
collected relevant administrative data, adding in the
randomized distribution had an extremely low marginal
cost. The randomized trial will allow the city to compare
the outcomes for officers with cameras to those without
cameras to answer important questions about the
technology’s effectiveness. Support from BIT through What
Works Cities this year will facilitate additional projects.
Yokum said that even doing a small, concrete project,
such as testing two subject lines for an email, can inspire
departments to come back with more ambitious ideas
for ways to apply the same methods to other areas. The
goal of the team is to support talented employees in all
departments and raise the city’s collective capacity to
use evidence to drive policy. Although it is still relatively
new, The Lab @ DC demonstrates promise to scale to an
evidence-driven District government.
Act: In the last 12 months, has your local government shifted funds away from a practice, program or policy that, through rigorous data analysis and evaluation, has consistently failed to achieve desired outcomes toward a more effective and efficient practice, program, or policy?
Last year, Jackson, Mississippi, faced a challenge familiar
to many cities: a need to make significant budget cuts
while trying to preserve jobs and maintain service delivery.
Because of Mayor Tony Yarber’s commitment to data
and the city’s prior achievements in developing an open
data portal and launching a performance management
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 9
program, the solution was obvious: turn to data to
restructure and repurpose funds.
Beginning in May 2016, four months before the budget
season began, Jackson began a comprehensive effort
to analyze its spending, programs, and results to identify
opportunities for efficiencies. The team responsible
consisted of directors, deputy directors, executive staff,
fiscal officers, and on-the-ground support workers. This
team worked with GovEx through What Works Cities for
technical assistance in data analysis.
With the help of GovEx, the city analyzed its budget in
comparison to comparable cities and identified areas
of disproportionate spending. GovEx also trained every
departmental data coordinator and fiscal officer in how to
analyze and visualize their own data with Tableau to enable
ongoing data use. The city then used this work to analyze
each department’s programs more granularly, which
leveraged the city’s JackStats performance management
framework to identify which were producing results aligned
with the city’s priorities.
The city made changes in many departments based on
the analysis. The Human & Cultural Services Department
merged low-performing senior and child care centers
with higher-performing ones. In other departments, staff
positions related to underperforming programs were
repurposed to higher-impact areas in order to avoid
layoffs. The city also looked at departmental structures
to identify existing functions that would be more efficient
under the purview of another department, such as moving
tree and limb removal from Parks & Recreation to Public
Works, which owned the necessary equipment.
Mayor Yarber and his budget team saw record turnout at
community outreach events related to the budget. Basing
decisions on the data helped the Mayor and his senior
staff have difficult conversations with departments and with
the community. The dashboards and visualizations that
the budget team used are available to the public and all
departmental employees, ensuring that the data behind
the decisions are transparent.
Justin Bruce, Director of Innovation and Performance,
emphasized the progression of work with data that
developed the capacity that made these efforts possible.
He said, “Open data allowed us to take data, clean it,
work with it, and actually track progress and performance.
JackStats helped us look at the data at a more granular
level to show us why and how we are meeting goals.” The
latest effort, he said, “allowed us to take performance
management to a different level, not just looking at what
we are doing but how efficiently and effectively we are
accomplishing our goals.”
Jackson was able to cut its budget by 7.6% with this
process. The city went from a $14 million deficit in 2014 to
a $6 million surplus this fiscal year, in addition to creating
a new reserve fund. The city plans to continue its priority-
based budgeting and repurposing work in the same way
going forward. Bruce said, “Every time we touch an issue
that has data to guide the situation, we’re always going to
do an analysis of that data and always going to determine
what’s most effective and efficient and, ultimately, what’s
best for our citizens.”
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 10
The stories above illustrate some of the many ways that
leading cities are leveraging the power of data and evidence.
By objectively recognizing successes like these and providing
a roadmap for critical data practices, What Works Cities
Certification will enable even more cities to improve the
way they work and deliver better services to their residents.
For the first time, an objective organization has produced
comprehensive and reliable criteria for high performance
that will produce a roadmap to operational excellence for
mayors aspiring to improve the quality of life in their cities.
This brief was written in conjunction with Harvard Kennedy
School’s Katherine Hillenbrand, Project Manager; Eric
Bosco, Research Assistant/Writer; and Chris Bousquet,
Research Assistant/Writer.
About the Author
Stephen Goldsmith is the Daniel Paul Professor of the
Practice of Government and the Director of the Innovations
in American Government Program at Harvard’s Kennedy
School of Government. He currently directs Data-Smart
City Solutions, a project to highlight local government
efforts to use new technologies that connect breakthroughs
in the use of big data analytics with community input to
reshape the relationship between government and citizen.
He previously served as Deputy Mayor of New York and
Mayor of Indianapolis, where he earned a reputation as
one of the country’s leaders in public-private partnerships,
competition, and privatization. Stephen was also the chief
domestic policy advisor to the George W. Bush campaign
in 2000, the Chair of the Corporation for National and
Community Service, and the district attorney for Marion
County, Indiana from 1979 to 1990. He has written The
Power of Social Innovation; Governing by Network: the New
Shape of the Public Sector; Putting Faith in Neighborhoods:
Making Cities Work through Grassroots Citizenship and
The Twenty-First Century City: Resurrecting Urban America;
and The Responsive City: Engaging Communities Through
Data-Smart Governance.
To apply for What Works Cities Certification, please visit whatworkscities.bloomberg.org/certification.
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 11
Appendix: Certification Criteria
Technical Assistance Framework
Open Data Data Governance Performance Analytics
Results-Driven Contracting Low Cost Evaluations Repurpose for Results
The What Works Cities Standard defines how local governments can create a strong foundation for the effective use of data and evidence. The Standard’s four components—Commit, Measure, Take Stock, and Act—build on each other to help cities understand and invest in what works:
The What Works Cities Standard
What Works Cities leaders make powerful, public commitments to achieving better results for their residents by using data and evidence when making budget and policy decisions;
What Works Cities leaders collect and use data and tools to measure progress and engage residents along the way;
What Works Cities leaders consis-tently review and reflect on the data and evidence they have to learn and make improvements; and
What Works Cities leaders use data and evidence to inform major decisions and take action to improve outcomes.
Commit Measure Take Stock Act
COMMIT
Does your local government have a codified open data policy?
Does your local government’s open data policy call for regular maintenance and at least an annual proactive release of government data online?
Does your local government’s open data policy require a process to ensure data quality and usability (i.e. Quality Assurance process, publication of metadata, searchable)?
Does your local government’s open data policy establish a governance structure that calls for actionable steps for city staff and oversight authorities to follow to see the policy through to implementation?
Does your local government’s open data policy require periodic review for potential changes to the open data policy and system?
Does your local government have a data governance practice to ensure data quality and usability (i.e. Quality Assurance process, documentation of metadata)?
Does your local government classify data according to sensitivity and need for protection?
Has your local government defined and made publicly available time bound, measurable citywide strategic goals (e.g., reduce homicide by 20% in three years)?
Does your mayor or chief executive publicly commit to strategic goals and progress toward them?
Does your local government have a policy or ordinance establishing a performance management program for the city (e.g., Stat, performance measurement, etc.)?
Does your local government have a policy or ordinance establishing evaluation requirements for city-funded practices, programs, and/or policies?
Does your local government’s policy require at least an annual evaluation for the largest city initiatives programs, and policies? Does your local government’s policy require an evaluation budget for budgetary investments?
Does your local government have a policy or ordinance establishing the review and modification of practices, programs, and/or policies, through rigorous data analysis and evaluation, that have consistently failed to achieve desired outcomes?
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
WHAT WORKS CITIES March 2017 | 12WHAT WORKS CITIES 2
MEASURE
TAKE STOCK
The What Works Cities Standard
Does your local government have an open data portal (i.e. a website for making electronic data records accessible in whole or in part to the public) and routine processes for adding new / updating published data?
Does your local government have a written process to determine the release of open data?
Does your local government use (where they exist) civic data standards when publishing open data?
Does your local government maintain a comprehensive data inventory?
Has your local government established or adopted data standards (e.g., address and date formats, preferred geospatial projections)?
Does your local government publish progress on city goals on at least an annual basis (e.g., annual report, update to city’s strategic plan, etc.)?
Does your local government measure outcomes, impacts, and/or cost-effectiveness of key procurements, contracts, and/or grants? (i.e. monitor performance data in real-time and troubleshoot with contractors to achieve the goals of the contract and/or grant)
Does your local government have publicly available baseline evaluation standards or evaluation protocols to protect rigor of city-funded evaluations?
Does your local government have a designated person or team responsible for managing data?
Does your local government have a designated person or team responsible for performance management?
Does your local government convene a performance management program (i.e. Stat meetings)? Does your local government have a set schedule for performance management or Stat meetings?
Does your mayor or chief executive as well as department commissioners regularly attend performance management or Stat meetings?
Does a senior official with budget and decision-making authority chair these meetings?
Has your local government selected specific performance measures as key indicators to highlight and visit annually?
Does your local government’s performance management program collect and store outcomes and performance data on city contracts?
Does your local government have a dedicated person or team responsible for strategically managing the city’s portfolio of most important procurements that are due in the upcoming year?
Is the procurement and contracts function organizationally directly below the local government manager or mayor?
Does your local government define strategic objectives and desired outcomes for each key procurement?
Does your local government structure the procurement and contract process (including selecting the appropriate contract type) to incorporate incentives and align to strategic goals?
Does your local government actively manage ongoing key contracts / grant? That is, does your local government monitor performance data in real time and trouble-shoot with contractors to achieve the goals of the contract or grant, as needed?
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
WHAT WORKS CITIES 2WHAT WORKS CITIES March 2017 | 13
TAKE STOCK
ACT
The What Works Cities Standard
Does your local government have a written process that calls for the public release of data that is relevant to stated city/department goals and objectives, fundamental services, or core mission?
Does your local government have a written process for providing a means for public data requests and release of data that is responsive to residents’ requests?
Does your local government make future contracting decisions based on a contractor’s past performance?
Does your local government apply results-driven contracting strategies for your five most important (either tying to high priority goals or representing large dollar amounts) contracts or procurements?
Does your local government have an evaluation system or scorecard for key procurements, contracts, and/or grants that facilitate comparison of outcomes across contractors to determine which contractors are most effective?
In the last 12 months, has your local government initiated low-cost or randomized evaluation of priority city programs or services in 5 of the city’s largest departments and/or programs? In the last 12 months, has your local government allocated budget specifically designated for evaluation as a condition or sign-off for new projects? In the last 12 months, has your local government used the results from low-cost or randomized evaluations to improve city programs or processes? In the last 12 months, has your local government used rigorous data analysis and evaluation to publicly identify practices, programs and/or policies that have consistently failed to achieve their desired outcomes? In the last 12 months, has your local government shifted funds away from a practice, program or policy that, through rigorous data analysis and evaluation, has consistently failed to achieve desired outcomes toward a more effective and efficient practice, program or policy?
Has your local government communicated the decision to shift funding based on practices, policies, and/or programs that, through rigorous data analysis and evaluations, are consistently failing to achieve desired outcomes to the public (e.g., residents, customers, elected officials)?
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
Does your local government have a designated person or team responsible for managing evaluations?
Does your local government have a publicly available protocol or process for conducting external research and evaluation projects (i.e. data sharing agreements, IRB-style internal review process, etc.)?
Does your local government have senior-level managers empowered to repurpose funds from practices, programs and/or policies that, through rigorous data analysis and evaluation, have consistently failed to achieve desired outcomes?
Does your local government have a written process for determining what action should be taken when a practice, program or policy has consistently failed to achieve its established outcome-based performance targets?