Upload
socratainc
View
217
Download
0
Embed Size (px)
Citation preview
8/13/2019 [Socrata] Open Innovation - Volume 2
1/21
Designing the Future:
Code for America
Makes an Impact
Data-Driven
Government
in Action
Codelescence:
Engineering Comes
of Age
Socrata
2013 Highlights
WINTER 2014
Fixingthe Federal ITProcurement Process
8/13/2019 [Socrata] Open Innovation - Volume 2
2/21
8/13/2019 [Socrata] Open Innovation - Volume 2
3/21
OPEN DATA
INFOCUS
OPEN INNOVATION WINTOPEN INNOVATION WIN TER 2014
OPEN DATAI
In todays world o data standards, we
are operating in pre-mainstream chaos,
similar to that which existed with
electricity in Edison and eslas time.
Te potential or data-as-a-uel exists as
a natural resource in much the same
way as magnets and copper did in the
industrial revolution. As this next wave
o economic opportunity takes root, the
same kind o arguments are being made.
GREAT THINGS ARE POSSIBLE
WITH OPEN DATA
We benefit rom technical standardsdaily. Tey allow drivers to purchase gas
rom any station without ear o using
incompatible uels. Tey allow a text
message rom a Verizon phone to be
transmitted to an A& phone. Data
standards operate in a similar manner,
but their use within modern applications
is relatively new. Te most popular one
in use today is the General ransit Feed
Specification (GFS). GFS allows users o
Google Maps to know when t he next bus is
going to arrive within participating cities.
Looking to evolving data standards that
deserve greater adoption, a great example
at the civic level is HouseFacts. Tere
are many businesses that help people
buy and sell homes (e.g. rulia, Zillow,
most commercial banks, etc.), but the
inormation on the saety and health o
those homes is messy and reported in
Why We NeedOpen DataStandards
Right NowBy Ian Kalin
Socrata Director of Open Data
As the open data movement gains
momentum, more organizations, businesses,
and citizens are looking to share data and
collaborate on projects. What is essential
to them doing so? Data standards. Data
standards are a topic that deserves our
communitys full attention right now. The
government innovation movement must
address their importance across industries,
across borders, and even between
departments.
THE DIFFERENCE
STANDARDS MAKE
First, lets discuss the importance o
standards in amiliar industries. Imagine
i the electrified world we live in today
had no standards; you might have to
replace your toaster every time you move
to a new house. When electricity first
made its way into the households o the
world, standards did not yet exist. o
that point, two o the worlds greatest
inventors, Nikola esla and Tomas
Edison, became engaged in an epic debate
between alternating-current versus direct-
current. Each argument had strengths and
weaknesses but esla made the innovative
choice to back his argument up with
household appliances, guaranteeing his
victory in the debate and establishing our
universal outlets today.
very different ormats. Te solution is
a uniorm ormat or reporting things
like asbestos, pest inestations, and
even abusive landlords. As standards
like HouseFacts are adopted by city
governments, businesses can aggregate the
data that most cities are already collecting
and integrate that inormation into the
websites people are using to make housing
decisions. Te map or standard adoption
becomes a map or business growth.
Te challenge lies in developing t
standards. Development can be d
because o the complexity o the
reflects the world in which it is cr
Te maintenance alone o engine
standards is hard, as demonstrate
huge network o organizations th
with standards, like the Internati
Organization or Standardization
the American National Standard
(ANSI), and UL.
As open data moves into our mai
world, making it as available as p
a consistent, efficient manner, is
encourage you to engage in the d
o defined data standards so that
open data truly serves t he world
who own it.
As open data moves into our mainstream
world, making it as available as possible, in a
consistent, efficient manner, is essential.
https://developers.google.com/transit/gtfs/http://housefacts.me/http://www.ansi.org/http://www.ul.com/global/eng/pages/http://www.ul.com/global/eng/pages/http://www.ansi.org/http://housefacts.me/https://developers.google.com/transit/gtfs/8/13/2019 [Socrata] Open Innovation - Volume 2
4/21
(polynomial trend line)
Data Served by Socrata 2011-2013
2011 2012 2013
OPEN INNOVATION WINTOPEN INNOVATION WIN TER 2014
OPEN DATAIPEN DATAIN FOCUS
Bigger, Faster,Broader: HowOpen Data UseIs Changing
y Ben McInnis
ocrata Director of Product
magine a state that tracks K-8 student consumption
f free and reduced lunch and puts that data
ongside test scores and attendance information.
magine this data is then summarized in an internal
ashboard and sent to teachers via a mobile app.
this scenario, millions of data points updated
egularly could help with resource allocation on a
onthly, weekly, or even daily basis.
sites, and decision support systems or
government. says Socrata VP o Product
Sa Rabah.
Businesses, governments, and citizens
now use open data on a daily basis, and
they want to look at as much o it at
once as possible. For example, the City
o Chicago offers a traffic dataset t hat is
more than one million rows long. And,
San Franciscos app showing parking
availability is updated in near-real-time.
Having hosted millions and millions o
rows o public data since 20 07, Socratahas seen data use change dramatically,
particularly over the last two years. Here
are three o the strongest trends weve
noticed.
BIGGER DATA
wo years ago, the average dataset stored
on Socrata was under 10,000 rows. odays
common datasets all somewhere between
10 - 40 million rows. Socrata stores
datasets as large as 100,000,000 rows and,
by end o 2014, we expect to store datasets
o 1 billion rows or more.
As data scales in size, two interesting
things happen. First, data becomes
more useul or indirect and unoreseen
purposes because, within larger datasets,
common keys are more likely to exist.
Second, with more and more correlated
records across datasets, the need to
precisely clean and ormat data decreases.
Because o these characteristics,
analysis o large datasets can ofen yield
surprising and proound results. For
example, the City o Chicagos analytics
team discovered a relationship between
streetlight outages and petty thef--
correlations that were always latent within
their massive 311 and crime datasets.
While the connection might have seemed
obvious, having the data to clearly
understand this relationship helped the
city strategically address the problem.
FASTER DATA
High-requency data (data thats updated
very ofen) has seen similar growth. In
the last two years, as data rom sensor
networks and operational systems has
been published as open data, the number
o real-time datasets Socrata hosts has
grown more than 2,000 percent. (See
chart above.)
While static data is useul primarily or
archival and analytical purposes, real-
time or streaming data can be leveraged
as an input into other systems. Snowplow
locations can be ed in real-time to
citizens and the media, bus locations and
traffic signal state inormation can be
synchronized to optimize traffic flows,
and less obvious connections can drive
business efficiencies. A coffee vendor
might synchronize the preparation o resh
coffee and warm pastries to correspond
with the anticipated arrival o a commuter
train - not the scheduled arrival, but the
arrival time according to data coming
rom the tra ins computer system.
By transorming data rom a historical
record to a dynamic eed, it can be uti lized
on an on-going basis to make programs
and decision-making more efficient.
MACHINE-TO-MACHINE DATA
Data is also increasingly accessed, not by
humans, but by systems and applications
via application programming interaces
(API). Tey make data assets useul by
integrating them as an input or other
systems. For example, a new generation o
connected and sensor-enabled municipal
water systems provide data on water usage.
Tis inormation lets citizens understand
and manage their consumption, p
officials accurately plan or capac
needs and seasonal variance, and
personnel pinpoint leaks and ser
interruptions.
A study by technology market re
firm Hurwitz & Associates titled
Benefits o APIs in the App Econ
ound that organizations offering
as opposed to those that didnt, in
customer reach by 70 percent, nu
apps created by 50 percent, and n
mobile platorms supported by 58
oday, nearly every dataset is acc
via API and many thousands o t
per day. Socrata is the largest pro
government data APIs in the wor
more than 77,000 datasets availab
via API.
And, the total volume o data ser
Socrata via API has grown more
1,500 percent in the last two year
explosion in API usage across the
data movement comes rom inno
citizen apps like WasMyCarowe
app uses public data APIs to tell m
who return to their parking spot
no car, i their car wa s towed and
where.
FROM ARCHIVE TO FUEL
As the size o datasets grows, req
o dataset updates increases, and
machine-to-machine interaction
datas useulness is being tapped
novel uses and reuses are coming
light. Publishing data or the sak
transparency is a noble, yet short
goal. In the era o big, ast, useul
we are challenged to do more.
This way o tracking and
reporting on school programs
via web and mobile apps is just
one example o the many that
will emerge as government organizations
offer more public data as open data.
Since its ounding in 2007, Socrata has
seen open data move in larger quantities,
aster, and to more places than ever beore.
Were evolving past the cat alog phase o
open data and into the data-as-a-platorm
era, where value is measured by the real-
time flow and distribution o data th rough
application ecosystems, consumer internet
http://hurwitz.com/recent-research/item/web-api-study-the-benefits-of-apis-in-the-app-economyhttp://hurwitz.com/recent-research/item/web-api-study-the-benefits-of-apis-in-the-app-economy8/13/2019 [Socrata] Open Innovation - Volume 2
5/21
OPEN INNOVATION WINTOPEN INNOVATION WIN TER 2014
OPEN DATAIPEN DATAIN FOCUS
Te Importanceof Engaged OpenData Advocates
y David Eaves
pen Innovation Expert
uring my 2011 keynote at Open Government Data
ampI talked about how the open data movement was
t an inflection point. For years we have been on the
utside, yelling that open data matters. Now we are
eing invited inside and we have a great responsibility
o be of service. Once you have world leaders talking
bout things like a G8 Open Data Charteryou are no
nger on the fringes - not even remotely.
initially. From the cell phone to t he car to
the printing press to open source sofware,
all these inventions have helped billions
o people, but they did not distribute
themselves evenly, especially at first.
So the question cannot be reduced to
will open data empower the empowered,
but to what degree, and where, and with
whom? Ive seen plenty o evidence where
data has enabled small g roups o people to
protect their communities or make more
transparent the impact (or lack there o)
o a government regulation. Open data
expands the number o people who canuse government inormation or their
own ends this, I believe is a good thing
but that does not mean we shouldnt be
constantly looking or ways to ensure t hat
it does not reinorce structural inequity.
PROMOTING THE IDEA OF DATA
AS A PLATFORM
Some o the issues around usability Ive
addressed above in the accessibility piece
or portals that genuinely want users,
the axis o evolution is pointed in the
right direction with governments and
companies like Socrata trying to embed
more tools on the website to make the data
more usable.
I also agree wit h a point by Proessor
Kitchin that, rather than creating a
virtuous circle, poorly thought out and
launched open data portals will create
negative doomloops in which poor
quality data begets little interest which
begets less data. However, the problem is
bigger than that.
One o the main reasons I have been an
advocate o open data is a desire to help
citizens, nonprofits, and companies gain
access to inormation that could help
them with their missions. I also wanted
to help change the way governments deal
with their data, so that they can share it
internally more effectively. I ofen cite a
public servant I know who had a sum mer
intern spend three weeks surfing the
national statistical agency website to find
data they knew ex isted but could not find
because o terrible design and search. A
poor open data site is not just a sign that
the public cant access or effectively use
government data; it usual ly suggests that
the governments employees cant access oreffectively use their own data. Tis is ofen
deeply rustrating to many public servants.
Tus, the most important outcome created
by the open data movement may be that
government organizations, save or those
in the intelligence community, realize that
they are not comortable with using data
to drive decisions. Getting governments
to think about data as a platorm (yes,
Im a an o government as a platorm or
external use, but above al l or internal use)
is, in my mind, one way we can enable
public servants to gain better access to
inormation. Adoption o this principle
will also, in many cases, obviate the need
or costly solutions rom huge vendors
(like SAP and Oracle), whose $100 million
dollar implementations ofen silo off data,
rarely produce the results promised and
are so obnoxiously expensive it boggles
the mind.
Te key to all this is that open data cannot
be something you slap on top o a big I
stack. I try to explain this in my blog post
Its the Icing Not the Cake about how
Washington, DC was able to effectively
launch an open data program so quickly
(which was, apparently, so effecti
bringing transparency to procure
data the subsequent mayor rolled
Te point is that governments ne
thinking in terms o platorms i
the long term open data is goin
And it needs to start thinking o
the primary consumer o the dat
being served on that platorm.
My main point is thi s: lets not pl
edges and merely define this chal
as one o usability. It is a much bi
problem than that. I we get it wr
then the big government vendors
inertia o bureaucracy win. I we
right, we could potentially save ta
millionswhile enabling a more
effective, and responsive governm
I try hard to be critica l advocate
data one who engages the risks
challenges posed by open data. I
perect and balancing these two g
advocacy and a critical v iew is
but I hope it is how we in the ope
movement see our role.
WHY IMPROVING DATA LITERACY
IS IMPORTANT
We must address the reality that data, even
while open to all, can be used to by the
most powerul to gain more power. Tere
are definitely cases where data can serve to
urther marginalize at-risk communities.
For example, we should never publish
publicly the locations o womens shelters
or worse, the list o amilies taking reuge
in them.
Tere are two things that g ive me some
hope in this space. Te first is that,
when it comes to open data, the a xis ocompetition among providers usually
centers on accessibility. For example,
the Socrataplatorm (a provider o open
data portals to government) invests heavily
in creating tools that make government
data accessible and usable to the broadest
possible audience. Tis is not a claim that
all communities are being engaged and
that a great deal more work cannot be
done, but there is a desire to show greater
use, which drives some data providers to
find ways to engage new communities.
Te second is that i we want to create a
data literate society and I think we do,
or reasons o good citizenship, social
justice, and economic competitiveness
we need the data first or people to learn
and play with. One o the best ways to help
people become data literate is to give them
more interesting data to play with. We did
not build libraries afer everyone knew
how to read, we built them beorehand
with the goal o having them as a place that
could acilitate learning and education.
Tere are also things that ofen depress
me. I struggle to think o technologies that
did not empower the empowered at least
Rob Kitchin, Proessor o
Geography at the National
University o Ireland at
Maynooth, recently wrote
a list o open data critiques or his blog
Programmable City. His post inspired
me to remind the open data community
particularly the advocates - o our
responsibility to take part in the debates
around open data, right now. We need to
engage in the discussions on a number o
topics i we want open data to reach its ull
potential or effecting positive change in
the world. Specifically, I will address two
critiques that Proessor Kitchin raised:
using data to empower the less powerul
and how to improve utility and usabilit y o
that data.
One of the main
reasons I have bee
an advocate of op
data is a desire to
citizens, nonprofit
and companies gaaccess to informat
that could help th
with their mission
http://eaves.ca/2011/10/21/the-state-of-open-data-2011/http://eaves.ca/2011/10/21/the-state-of-open-data-2011/http://eaves.ca/2011/10/21/the-state-of-open-data-2011/http://www.international.gc.ca/g8/open_data-donnees_ouvertes.aspx?lang=enghttp://eaves.ca/2011/07/12/its-the-icing-not-the-cake-key-lesson-on-open-data-for-governments/http://www.socrata.com/http://www.socrata.com/http://eaves.ca/2011/07/12/its-the-icing-not-the-cake-key-lesson-on-open-data-for-governments/http://www.international.gc.ca/g8/open_data-donnees_ouvertes.aspx?lang=enghttp://eaves.ca/2011/10/21/the-state-of-open-data-2011/http://eaves.ca/2011/10/21/the-state-of-open-data-2011/8/13/2019 [Socrata] Open Innovation - Volume 2
6/21
OPEN INNOVATION WINTEOPEN INNOVATION WI NTER 2014
OPEN DATAIPEN DATAIN FOCUS
Open Datan Europe
Leaps Beyondransparency
y Erika Smith
crata VP of Worldwide Markets
he open data movement is thriving in many parts
f Europe, including well-established programs
the Netherlands, the United Kingdom, and
candinavian countries like Sweden and Norway.
ut, in response to recent questions like, What
ave we achieved and what can we do with all
f this data? the conversation has turned from
pen data for transparencys sake to open data to
rengthen the economy.
In act, today, the United Kingdom
hosts one o the most mature open data
programs in the world. From this vantage
point, UKs thought leaders are talking
more and more about the t remendous
economic and social potential o open
data, especially in machine-readable,
standardized ormats.
In late October 2013, Open Data Institute
(ODI) ounders Sir im Berners-Lee
and Sir Nigel Shadbolt announced the
creation o a global network o nodes
where programs aligned with ODIs
principles o openness and economic
innovation would be established. As
part o the announcement, they were
quoted as saying, Te best way that open
becomes the new deault is demand:
rom businesses and organizations, both
public and private, rom individuals and
corporations.
Berners-Lee and Shadbolt assume that
open data will be put to use supporting
new businesses. Te ODI, a not-or-profit
organization that is just over one year old,
has so ar helped set up more than a dozen
open data-based startup companies in
the UK.
Socrata became an official partner o
ODI in the all o 2013. ODI lead Gavin
Starks says, We love Socratas view o
open data as uel or new businesses. We
delivery within Europe, Socrata r
partnered with Microsof to mak
Socrata Open Data Portal availab
Microsof Azure platorm. Azure
number o data centers on the co
Socrata Senior Site Reliability En
Paul Paradise noted, As a growi
company, we scale internationall
better by partnering with an esta
trusted, international data-hostin
platorm like Microsof Azure.
MORE BUSINESS IN THE FU
While transparency is still a centgoal or the open data movement
traditionally transparency-ocuse
organizations, like the Open Gov
Partnership (OGP), acknowledge
economic power o open data. Be
its Open Government Partnershi
Summit in London in October, th
announcedthat 37 countries had
commitments to, among many e
radically open up government d
boost entrepreneurship, growth a
accountability.
Open data is uel or innovation
world will never be the same, now
we can so easily take public data
it to work to make peoples lives b
says Merritt. Businesses will ine
put the public data shared to wor
economy. Socratas job is to make
it is easily accessed and in useul
want to collaborate with them to promote
the creation o new products and services
based on open data.
Socrata CEO Kevin Merritt presented
on a panel at the Open Data Summit
with the title, Open or Business Te
Commercial Impact o Open Data.
Merritt commented on the growing
interest in economic growth through
open data, saying, In the UK, and all o
Europe, people are asking, Open data, so
what? And, the ODI and other incubators
o new businesses help demonstrate that
innovation, new jobs, and better services
are possible when non-private data is easy
to access.
FASTER MOVEMENT OF DATA TO
SUPPORT INNOVATION
With the growing list o businesses relying
on government open data, the speed
and requency with which governments
deliver that data has become more
crucial. For example, Spend Networkis a
startup supported by ODI that uses open
spending data to create new insights or
Government and its suppliers. Spend
Networks services are only as useul and
accurate as the data it gathers.
Anticipating the greater demands on
datacenters across Europe and, in an effort
to improve the speed o its existing data
UK AIMS FOR ECONOMIC
GROWTH
In the all o 2010, when the UK launched
a searchable database o business plans
or government departments, Prime
Minister David Cameron was quoted as
saying, We want to be the most open
and transparent government in the
world. Since then, getting non-sensitive,
non-personal data online has become
something British citizens expect.
With the growing list of businesses relying ongovernment open data, the speed and frequency
with which governments deliver that data has
become more crucial.
http://www.theguardian.com/technology/2013/oct/29/web-inventor-open-data-institute-new-global-networkhttp://www.theguardian.com/technology/2013/oct/29/web-inventor-open-data-institute-new-global-networkhttp://www.theguardian.com/technology/2013/oct/29/web-inventor-open-data-institute-new-global-networkhttp://www.opengovpartnership.org/ogp-summit-statementhttp://www.spendnetwork.com/index/http://www.spendnetwork.com/index/http://www.opengovpartnership.org/ogp-summit-statementhttp://www.theguardian.com/technology/2013/oct/29/web-inventor-open-data-institute-new-global-networkhttp://www.theguardian.com/technology/2013/oct/29/web-inventor-open-data-institute-new-global-network8/13/2019 [Socrata] Open Innovation - Volume 2
7/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2014
OPEN DATAIPEN DATAIN FOCUS
Making the wo-Way Street of OpenData a Reality
y Chris Whong
ocrata Data Solutions Architect
Department o Inormation echnology
and elecommunications (provider o
the ootprints data) and are launching
a community project to systematically
import the citys treasure trove o
building data into OSM. Te project
was announcedin late September.
LEGIONS OF GEOSPATIAL
ANALYSTS
So whats in it or the city? Te potential
or updates that only a system like OSM
can provide. I OSM users see something
wrong, they can fix it. Maybe a buildingootprint is misaligned, or maybe the
building doesnt exist anymore. Te city
receives a daily update o changes to
the building data, and can review those
changes. I they are legitimate, DoI
can use these changes to gu ide updates to
their own master database, making it more
accurate and up to date. Its as i the cit y
has legions o geospatial analysts quality-
checking their data and sending updates!
Alex Barth, Data Lead at the web mapping
company MapBox and OSM Advocate, has
been a key organizer o the N YC OSM
collaboration, and has been working on
the idea since early 2012. Te data was
already publicly available back then, but
carried a license that was incompatible
with OSM. NYCs Open Data Law, passed
in March 2012, cleared up the licensing
issue and provided the way orward.
o Barth, the project is not simply about
buildings, but is an experiment and
learning experience about the impact o
community-driven projects like OSM. Its
a data improvement effort that has positive
side effects and really lets us grow t he
community. Te longer-term vision goes
beyond OSM or even geodata, and hopes
to redefine open data publishing: Tis is
about an open data commons, a single space
in which government and citizens interact.
Te first gathering o volunteer mappers
to work through the monumental task o
importing the citys data met in October.
Liz Barry, another leader in the NYC-OSM
collaboration, hosted the meeting at the
offices o the Public Lab in Brooklyn, and 22
community members showed up to help. Te
data was broken down into election districts,
and the team set out validating ootprints
against aerial imagery, checking geometries,
and correcting overlapping polygons.Existing attribute data in OSM could also
be merged with better polygons rom the
city data. Barry said the workflow is still
being vetted, and is not quite ready or ul l-
scale deployment. Te idea is that once the
workflow is perected, updates wont require a
physical meetup. Volunteer OSM users will be
able to import a chunk o t he citys building
ootprints whenever and wherever they can.
Te real un will begin when large amounts
o the data have been successully imported
and the city can report back about the volume
and utility o OSM-contributed changes. In
many cases, there may be more inormation
about a building in OSM than the city
maintains on its own, meaning the two-
way street o open data may not flow evenly
in both directions. Te OSM community
has ound a partner in DoI, and this
experiment will serve as an early model o
the power o citizens and activists to improve
government data.
OpenStreepMap allows users to view multiple regions,
from a birds-eye view of Europe to a zoomed in view of the
streets of Manhattan.
and can be just as easily undone. Users
may choose to update the map or many
reasons, rom just knowing more about
conditions on the ground than anyone
else, to improving the map or a specific
project such as an app.
What i a user needed some building
outlines that OSM didnt have yet? Tat
user could manually trace over the satell ite
imagery, pointing and clicking lots o
custom polygons into existence. But what
i they needed a whole town? What i
they needed New York City? Tey do, and
NYC has an open dataset or t hat. Te
citys detailed GIS database o building
outlines and point data is reely available
or download at data.cityonewyork.us.
While it will still take human effort to
import and veriy data or over a million
buildings, creating them manually would
be an unathomable and time-consuming
process. Leaders in the mapping tech
community have partnered with N YCs
Open data has by and
large been a one-way
conversation. Governments
produce public data and
ake it reely available, while citizens,
urnalists, researchers, and hackers
nsume it in whatever ways suit them.
ut, having more eyes on the data once it
released may be able to provide value
ck to the government, turning users
the data into a source o new data and
ality control. Tis is the experiment in
o-way open data that New York City is
oneering with OpenStreetMap.
penStreetMap (OSM) is the Wikipedia
Maps, where anyone can contribute
anges. (Yes, i theres a ootpath or bike
ail near your house that doesnt show up
mainstream web maps, you can literally
raw it into OSM, name it, and connect
to existing roads.) Like Wikipedia,
anges to the map are subject to quality
ntrol by the rest o the community,
https://www.mapbox.com/blog/nyc-and-openstreetmap-cooperating-through-open-data/http://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtmlhttp://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtmlhttp://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtmlhttps://data.cityofnewyork.us/http://www.openstreetmap.org/#map=5/51.500/-0.100http://www.openstreetmap.org/#map=5/51.500/-0.100https://data.cityofnewyork.us/http://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtmlhttp://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtmlhttps://www.mapbox.com/blog/nyc-and-openstreetmap-cooperating-through-open-data/8/13/2019 [Socrata] Open Innovation - Volume 2
8/21
OPEN INNOVATION WINTEOPEN INNOVATION WIN TER 2014
OPEN DATAIPEN DATAIN FOCUS
Open DataHad Better BeData-Driven
y Thomas Levine
Dada Artist
Weve been opening government data for some
me now. Without realizing it, weve amassed
ome rather rich data about how people publish
nd consume open data. With this data about
he use of open data, its possible to use data-
acked benchmarks, projections, and decisions in
ur open data strategies, and this can make our
pproaches to open data more systematic, logical,
nd obtainable.
urious about what the data behind open data can
each us? Read on.
Every dataset has basic, easy-to-extract
properties like the number o records
it contains, and the size it takes up on a
hard drive. We can come up with more
complicated inormation too, like the
number o missing values and the date o
the oldest record.
In addition, when a dataset is published
online, it creates metadata about t he
dataset and its use. Metadata provides
details like when it was first published,
who uploaded it, and how many times
people have downloaded it since then.
By collecting some o these simple
properties and metadata rom each
dataset, its possible to create a dataset
about the publication and use o other
datasets what I call a super-dataset.
DATASETS AS DATA POINTS
A super-dataset compiles inormation
about each dataset when it was
published, what kind o inormation it
contains, etc. into a single row, creating a
record (i.e. a data point) about that dataset.
With this setup, you can perorm all
kinds o unctions and analyses about the
publication and use o datasets: you can
look at how many datasets are on different
catalogs, how data is queried and reported,
what licensesdatasets have, and how many
there are o a certain category. Here are
some examples o analyses I have done.
Number of Datasets:I compared the
number o datasets on some various
government data portals that r un Socratas
open data platorm sofware. I ound that
New York City, Chicago, and the state o
Oregon offer the most datasets.
Licensing:I also used my super-dataset o
properties and metadata to look into this
question: What licenses do people apply
to their open data? I discovered that many
portals avor public domain or some orm
o open license, but most list no license at all.
Groupings of Datasets:By looking
at similarities in the titles, schemas,
and other metadata o datasets,
I determined what sorts o data
government organizations were putting on
their open data portal s and how different
datasets were related to each other.
METADATA AS DATA
Even though the word metadata contains
the word data, people dont typically
think o metadata as something to analyze.
Metadata is ofen invisible. I we could
see it, it might just look like background
inormation about web pages and their
contents. For example, Socratas open
data platorm displays metadata about
each dataset, such as community rating,
number o visits, number o downloads, etc.
When you use metadata to populate
records in a super-dataset, that metadata
becomes data that we can see, analyze, and
learn rom.
WHY THIS MATTERS
Tere are numerous case studies, by
organizations such as Code or America,
CKAN, Open Data Institute, about how
to open up government data, but these are
based strongly on personal experiences,
not precise, quantitative statistics. Tere
is comparatively littlework that uses data
to produce guidelines or best pract ices or
the opening o data.
Open data experts ofen talk about making
use o open data and building products
rom it, but a super-dataset accomplishes
something different. It reveals inormation
A visualization of the
datasets available on
data.cityofnewyork.us by
category, showing both
raw datasets, as well as
derivative charts, maps, and
filtered views.
about how people publish and us
data. It paints t he bigger picture
going on, whats working, and wh
Imagine what we can do with thi
backed understanding o our ope
We can find out what has been do
and what has worked, allowing o
publishers to plan their releases m
strategically. We can measure rel
strategies against solid, quantitat
statistics to make sure that they a
helping us achieve our goals. We
even use these findings to build p
that help people interact with op
Tese are just some o the possib
emerge when we think o data ca
datasets o datasets.
DATA, DATA, EVERYWHERE
We have data about open data, but it
always starts out in ormats that are not
convenient or this sort o study o open
data. So, we first need to turn this raw data
into a dataset. In my mind, a dataset is a
collection o things, with some consistent
properties describing each o the things.
(We ofen represent datasets as tables.)
Were going to treat each dataset a s a thing
inside our collection o many datasets.
http://thomaslevine.com/!/socrata-deduplicate/http://thomaslevine.com/!/socrata-deduplicate/http://thomaslevine.com/!/socrata-deduplicate/http://www.chriswhong.com/nycopendata/http://thomaslevine.com/!/open-data-licensing/http://thomaslevine.com/!/missouri-data-licensing/http://beyondtransparency.org/http://ckan.org/case-studies/http://theodi.org/case-studieshttp://thomaslevine.com/open-data/http://thomaslevine.com/open-data/http://theodi.org/case-studieshttp://ckan.org/case-studies/http://beyondtransparency.org/http://thomaslevine.com/!/missouri-data-licensing/http://thomaslevine.com/!/open-data-licensing/http://www.chriswhong.com/nycopendata/http://thomaslevine.com/!/socrata-deduplicate/http://thomaslevine.com/!/socrata-deduplicate/8/13/2019 [Socrata] Open Innovation - Volume 2
9/21
OPEN INNOVATION WINTEOPEN INNOVATION WI NTER 2014
FixingtheFederal IT
ProcurementProcess
By Kevin Merritt
Socrata Founder and CEO
Sometimes it takes a disasteror people to recognize the
need or change. Te botched
rollout o HealthCare.gov
something I would actually describe as a
systemic catastropheprovides a historic
opportunity to overhaul the ederal
government technology procurement
system, which is so clearly broken.
DIAGNOSING THE PROBLEM
Government technology projects have long
been characterized by significant delays,
mammoth cost overruns, and sofware
products that routinely underperorm.
But, beore the HealthCare.gov fiasco,
relatively ew people were aware o the
magnitude o the problems plaguing
the ederal I purchasing system. Now
that the story has made ront-page news,
everyone wants to know what went wrong.
Just as any good sofware engineer looks
to uncover the root causes o bugs in a
computer program, it is important to
meticulously diagnose the underlying
causes that continue to produce I
disasters at the ederal level.
SELLING TO THE FEDERAL
GOVERNMENT IS A BYZAN
PROCESS
Federal I procurement is overly
and opaque. Te process is ull o
rules and oppressive costs that di
many technology providers rom
the market. Te Federal Inorma
Security Management Act (FISM
a prime example. Tis legislation
enacted at a time when applicatio
almost always hosted on premise
waterall methodology (not ag
the predominant approach to sof
development. imes have change
have learned a lot since then.
Yet, because FISMA is still in pla
vendors looking to bid on ederal
technology projects must comply
with outmoded inormation secu
requirements, like using tape bac
replicate files offsite. Tis is comp
incongruent with modern I bes
practices. More importantly, the
data loss in a single environment
tape backups is statistically much
than in a geo-redundant environ
where data is replicated across m
servers.
8/13/2019 [Socrata] Open Innovation - Volume 2
10/21
OPEN INNOVATION WINTEOPEN INNOVATION WI NTER 2014
INNOVATORS NEED NOT APPLY
Te net result is that many nimble,
innovative companies simply orego doing
business with the ederal government,
concluding that it is just too costly. Over
the years, this has created an environment
that nurtures mediocrity. oday, there
are a whole class o enterprise companies
that specialize in navigating the bloated
and broken procurement system. Most
o these businesses make their money by
selling consulting hours, hiring dozens
o subcontractors to do the work, and
overseeing the development o expensive,
customized technology solutions that
ofen do not perorm as advertised.
We admire startups because their core
competency is innovation. Tey establish
traction and momentum because they
invent novel solutions to old problems,
develop new technologies, or come up with
creative new ways o doing business. Te
irony in the ederal I space, however, is
that these behemoth systems integrators
(SIs) are wi nning business not because
THE GSA JUGGERNAUT AND
OTHER BURDENS
o bid on ederal government projects,
businesses must first be listed on the
General Services Administration (GSA)
schedule. Maintaining this listing is
practically a ull-time job, requiring
continuous updates, renewals, and, o
course, piles o paperwork. o keep up,
many smaller businesses are compelled
to hire consultants just to clear all o the
administrative hurdles.
In a very real sense, GSA requirements
operate like an extra tax on businesses,
siphoning profits rom sales to ederal
government agencies. In addition to this
tax, GSA vendors must offer their lowest
commercial price to the government, even
i the market va lue or their products is
significantly higher. Tis squeezes profits
urther, disproportionately affecting the
ability o smaller businesses to compete.
As i this were not burdensome enough,
any vendor that wants to sell to the ederal
government must be able to decipher the
Federal Acquisition Regulation (FAR),
the principal set o rules that govern the
ederal procurement process. Te FAR
is every bit as complex and unwieldy as
the ederal tax code, containing rules as
picayune as the width o the margins or
request or proposal (RFP) responses.
And these rules change constantly. Few
emerging technology firms have the
wherewithal or the patience to wade into
that kind o logistical labyrinth.
they have innovated. Rather, it is just the
opposite. Teir sole core competency
has nothing to do with technology or
innovation at all; their only real skill is
that they have perected the art o selling
to the government!
CARVING A PATH FORWARD
Government leaders need to introduce
a serious-minded reorm agenda i any
real progress is to be made in revamping
the ederal I procurement process. Tis
means directly applying the open data
ideals that the Obama administration has
generally espoused to the specifics o the
I purchasing process.
First and oremost, there needs to be ull
transparency into ederal contracts. All o
the details or each contractincluding
the amounts paid and the names o the
people who authorized each payment
need to be online in machine readable,
ully sortable, and searchable ormats.
Te ederal government also needs to
make it simple or the public to ollow
the flow o money and influence between
ederal government agencies requesting
vendor support and the contractors
bidding on projects. As a country, we
have made great strides in recent years
in increasing visibility into campaign
contributions. o weed out corruption
and improve efficiency, senior government
leaders need to bring that same level o
effort and scrutiny to the procurement
process.
Further, policy makers should en
providers o cloud and SaaS-base
technology solutions to discuss w
to modernize the ederal governm
inormation security requiremen
the wake o the a ilure o HealthC
there is an exciting opportun ity t
the government officials who cer
security mandates about modern
best practices.
WHO IS READY TO TAKE U
MANTLE OF PROCUREMEN
REFORM?
Te once-stodgy subject o eder
procurement has gone viral. In tu
this awareness has produced broa
consensus around the need or m
changea precious commodity t
should not be wasted. Right here
now, I am putting orth a clarion
civic leaders on both sides o the
the time is ripe to take up the ma
real procurement reorm. By unl
the ingenuity o sofware compan
that have previously been pushed
sidelines, you can help launch a n
technology innovation that impr
lives o Americans or generation
to come.
Government
leaders need
to introduce a
serious-minded
reform agenda if
any real progress
is to be made
in revamping
the federal IT
procurement
process.
We admire
tartups because
heir core
ompetency is
nnovation. They
stablish traction
nd momentum
ecause they
nvent novel
olutions to old
roblems, develop
new technologies,
r come up with
reative new ways
f doing business.
8/13/2019 [Socrata] Open Innovation - Volume 2
11/21
I was very impressed to see the level of
civic innovation and ownership from the
Code Michigan hackathon participants.
Many coders I spoke to had never been to
a hackathon before and were very proud
to share how their products were going to
improve their city and many others.
Ian Kalin
Director of Open Data for Socrata
at Code Michigan
It is so amazing to see in a
city that has such a diverse
set of problems that we can
unite to create a diverse set
of solutions.
Beth Blauer
Director of GovStat for Socrata
at Hack for Change Baltimore
This is an amazing display of
the power of civic hacking and
what we can all do together.
Hannah Young
Program Coordinator of the National
CfA Brigade on the National Day of
Civic Hacking
For those of us who
work in government,
this feels like our own
national holiday.
Shannon Spanhake
City of San Francisco
Deputy Innovation Officer
on the National Day of
Civic Hacking
AWARDS
EVENTS
OPEN INNOVATION WINTEOPEN INNOVATION WIN TER 2014
SOCRATA2013 HIGHLIGHTS
13 was a big year for the open data movement. It was filled with
ckathons, datapaloozas, and hack nights. Leaders and innovators in the
ovement won awards and recognition. We here at Socrata were thrilled
be a part of it all. Here is a look back at some of th e top U.S. events
attended, awards our customers won, and favorite things we heard
ng the way.
Awards
Bloomberg Philanthropies
Mayors Challenge (Chicago)
Data Liberation Award
(NYC Dept of Health)
2013 Digital Counties Survey
Award (Montgomery County,
Snohomish County, King County)
State Program Innovation Award
(Oregon)
2013 Web 2.0 Award (Raleigh, NC)
2013 Best of the Web Winner
(Alameda County)
2013 Digital Government
Achievement Award Winner
(Alameda County)
2013 CSAC Merit Award
(Alameda County)
2013 Achievement of Excellencein Procurement Award
(Alameda County)
2013 Digital Cities Winners
(Boston, Seattle, Austin, Chicago,
Baltimore, Raleigh)
Events
ATX Civic Hackathon II
(Austin, TX)
Code Michigan
Code Across America
(Philadelphia)
Code Across America (
Vegas Hack
San Francisco Housing
Open Science (Mountai
EcoHackSF (San Francisc
Urban Data Challenge
Hackathon (San Francis
Data P2PU Course Spri
(Mountain View)
Code for Seattle
NC Data Jam (Raleigh)
Hack 4 Change (SeattleAlameda County Apps
NYC Data Week
SpoCode (Spokane)
Apps4Halifax
API Craft Conference(
API Strategies and Pra
Conference (San Francis
Code for America Sum
(San Francisco)
Code for L.A.
NC Datapaolooza (Rale
Hawaii Digital Governm
Summit
SXSW Eco Hackathon 2
(Austin)
Atlanta Govathon
Code for Oakland
International Open Da
(Washington, D.C., New
Seattle, Atlanta)
8/13/2019 [Socrata] Open Innovation - Volume 2
12/21
OPEN INNOVATION WINTEOPEN INNOVATION W INTER 2014
DESIGNINGTHE FUTURE:CODE FORAMERICA
MAKES ANIMPACTBy Alida Moore
Socrata Content Strategist
Before 2011, parents in Boston, MA struggled to figure out
which schools their children were eligible to attend. The
confusing process, involving a 28-page manual and a lottery
system, was a source of contention and, at times, violent
conflict among parents across the city. A turning point came
when Code for America (CfA), a non-profit organization
dedicated to using data and technology to improve lives,
turned their team on the problem.
8/13/2019 [Socrata] Open Innovation - Volume 2
13/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2014
Lauren Reid, Senior Public
Affairs Manager at CA,
explains how they built a
solution. Te CA ellows in
Boston developed an app where parents
could enter a ew simple data points, such
as address and age o child, and find out
quickly which schools were available as
options or their child, Reid says. Te app
was hugely successul. Te City o Boston
told us that this app, developed in just
three short months, would have taken the
city more than two years and two million
dollars to create -- had it gone through the
standard procurement process. ogether
with the City, were resolving a decades-
old problem using modern technology and
open data, and changing the conversation
between parents and the schools system
rom one o contention to one that is
positive and productive.
Code or Americawas ounded in 2009
by Jennier Pahlka, the current Deputy
Chie echnology Officer or the United
States. Te organization seeks to find
out what can happen when smart, savvy
researchers, developers, and designers
are deployed to cities across the nation to
help local governments unlock their data
to create solutions and deliver services to
their citizens.
THE FORMULA
Te Code or America ormula is simple.
Developers and designers commit a year o
their careers to helping a city government
in need o problem solving. In exchange
or a modest stipend, these participants,
called ellows, live in their assigned
cities, and use their skills to help move
government orward to meet the needs
o 21st century citizens.
CFA PROGRAMS
Code or America has developed our
programs to help urther their mission.
Te Fellowship:Code or Americas
flagship program, in which developers
and designers are matched with local
governments to transorm data into usable
orms or public improvement.
Te Brigade:Civic-minded volunteers
come together to orm brigades, bringing
grassroots efforts to data use and
transparency.
Te Accelerator:Provides financial and
logistical support to civic tech startups,
rom a $25K grant to mentorship and
networking opportunities.
Te Peer Network:A learning network or
government innovators who want to work
with other local governments to harness
the power o open data in their cities.
Ben Berkowitz, CEO and
Co-Founder of SeeClickFix,
presents at the Code for
America Summit 2011.
Top: Andrew McLaughlin,
self-proclaimed nerd,
Sunlight Foundation board
member, and CEO of Digg
and Instapaper, presents
at the Code for America
Summit 2011. Bottom: Laura
Meixell, 2013 Code for
America Fellow in
San Francisco.
FELLOWS IN ACTION
Matthew Hampel, 2012 CA Fellow
(Detroit), spoke to Socrata about his
experience as a ellow. One o his projects
or the city o Detroit was to create a
web and mobile app that would update
commuters about bus schedules. Hampel
told us about the app, called extMyBus.
Originally, bus data was tracked
manually through an ancient interace
and it wasnt available to public, Hampel
explained. So the city provided us with
access to their servers and we exposed the
data. With that, we built a text messaging
app that helps people figure out when their
buses are arriving. extMyBus has proven
quite popular. As o December 2013, the
app has served over 1.1 million users.
CA ellows have contributed 62 apps
across America so ar, rom helping
citizens navigate the public school system
to receiving text alerts when services, like
ood stamps, are about to expire.
We are on
the cuttin
edge of G
2.0 and c
hacking a
geeking.
http://codeforamerica.org/http://codeforamerica.org/8/13/2019 [Socrata] Open Innovation - Volume 2
14/21
OPEN INNOVATION WINTEOPEN INNOVATION WINTER 2014
Foley agrees. Your civic technical
ecosystem should overlap with
entrepreneurship. As part o that
ecosystem, our brigade influences the civic
I department by promoting open data.
We are on the cutting edge o Gov 2.0 and
civic hacking and geeking , Foley says.
You can expect to see more brigades pop
up in 2014, allowing or crossover and
partnership. As Foley explains, Your city
boundaries shouldnt limit innovation.
I, or example, you have an app t hat
catalogues the greenways in your city,
it should continue beyond city limits,
says Foley.
A LOOK FORWARD
In 2014, Code or America plans to add 31
new ellows to the ellowship program.
For people like Matt Hampel, who is
passionate about harnessing open data or
civic improvement, Code or America has
been a vehicle or positive change. Data
helps you design the uture you want to
see, Hampel explains. Data gives you the
power to make inormed decisions, and
inormed decisions are better decisions.
Code for Americaby Numbers
2011
19fellows deployed to Boston, Philadelphia, and Seattle
2012
26fellows sent to eight cities: Austin, Chicago, Detroit, HonMacon, New Orleans, Philadelphia, and Santa Cruz2013
27fellows embedded within nine communities: Kansas City
Kansas City, Mo.; Las Vegas; Louisville, Ken.; New York;
Calif.; San Francisco; San Mateo County, Calif.; South Be
Summit County, Ohio
2014
31fellows will join the Code for America program. Cities toannounced.
62apps created by CfA fellows over the years
p: Volunteers gather at
rigade meeting in San
ancisco. Bottom left: Former
y of Seattle Chief Technology
ficer Bill Schrier presents at
e Code for America Summit
11. Bottom Right: CfA will send
ore fellow to more cities than
er before in 2014.
BRIGADES IN ACTION
Some o Code or Americas most
passionate, committed volunteers live in
Raleigh, NC. Jason Hibbets and Chad
Foley are two o the Raleigh brigades our
co-captains. Each man volunteers hours o
time each month, outside o their day jobs,
to the brigade.
Being part o the Raleigh brigade allows
us to have an impact on loca l government
and our community, says Hibbets. He
believes that government transparency
in data is paramount to innovation.
Open data is the oundation o civic
entrepreneurship, Hibbets explain s.
Open data belongs to the people. Put
data in the right hands and apps that can
improve the daily lie o citizens can
be created.
Data gives you
the power to
make informed
decisions,
and informed
decisions are
better decisions.
8/13/2019 [Socrata] Open Innovation - Volume 2
15/21
OPEN INNOVATION WINTEOPEN INNOVATION WIN TER 2014
DATADRIVENGOVERNMENTIN ACTIONBy Beth Blauer
In the previous issue of Open Innovation, I
talked about the importance of fact-based
decision-making and how gut decisions
can be expensive and dangerous. For
governments to play a leading role in the
data revolution, performance measurement
and successful delivery are mandatory. This
truth is something I am passionate about.
My passion resonates from my experience
managing StateStat and the Delivery Unit in
Maryland and is affirmed in conversations I
have had with government leaders since.
Beth Blauer is a
leading expert
in implementing
stat programs in
government and is
Director of GovStat
at Socrata.
Fortunately, the open data
movement has resulted in
tools or data-driven decision-
making, perormance, and
delivery. Socratas product, GovStat,
helps local governments become more
transparent, engage citizens, and measure
progress against initiatives and goals. It
also allows stakeholders to collaborate
throughout the entire process on one
common platorm.
In Beyond Transparency: Open Data and
the Future of Civic Innovation1, I stressed
the idea that its not a matter o i data-
driven government can create the best
solutions to societys problems; its a matter
o how soon governments will embrace the
idea and reap the benefits. Using a tool li ke
GovStat enables governments to collect
and update data across departments,
build beautiul data visualizations, and
create both internal and citizen-acing
dashboards to track progress.
I would like to highlight three governmentorganizations that have chosen to use
GovStat as their primary perormance
measurement solution. I will share their
specific challenges in moving toward data
1 Code for America Press (2013)
transparency, why they chose GovStat as
their solution, and how each organization
plans to use GovStat to i ncrease efficiency
and accountability to their constituents.
TELLING THE STORIES
THAT MATTER:
COOK COUNTY, IL
In 2011, Cook Countybegan a program
to create a strategic plan around the use
o open data or decision-making and
communications. Andrew Schwarm, Chie
Perormance Officer o Cook County, was
the project head tasked with finding a
perormance measurement tool.
Te county was publishing our quarterly
reports in a PDF on our website, which
was ar rom best practice, Schwarm says.
Using a tool lik
GovStat enable
governments
to collect and
update data ac
departments, b
beautiful data
visualizations, create both int
and citizen-fac
dashboards to
progress.
http://beyondtransparency.org/http://beyondtransparency.org/http://beyondtransparency.org/https://performance.cookcountyil.gov/https://performance.cookcountyil.gov/http://beyondtransparency.org/http://beyondtransparency.org/8/13/2019 [Socrata] Open Innovation - Volume 2
16/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2014
We wanted to transorm our external
rormance reporting rom a static
DF to a dynamic open data web portal.
ovStat jumped out as the product that
ade the most sense or us.
n early adopter o the GovStat platorm,
ook County decided on a two-phase
plementation process. First, the County
plicated the PDF data and created
ports or each department. GovStat
owed us to take the data already
thered and put it on a more flexible,
er-riendly, open, and transparent
atorm, Schwarm says.
nce the data was updated and made
ailable to the entire organization,
ook County entered phase two o
plementation: using data to drive
cisions. [Data-driven decision-making]
now part o our culture a nd the way we
business, explains Schwarm. Further,
accurate data allows Cook County to tell
accurate stories. One goal the county
tracks is lowering sick leave or county
staff; Cook County hopes to reduce
employee sick leave to 5.2 hours per month
beore December 2013. On October 25,
2013, Cook County published a GovStat
report on the progress toward this goal,
declaring it on track.
Schwarm is excited to track other county
initiatives, including public saety and
healthcare goals. We plan to roll out one
goal per month and continue to report on
its progress, Schwarm says. He continues,
Te use o good, timely, accurate data,
especially or a government acing tough
fiscal situations, allows us to make
decisions and prioritize at a high level.
Citizens of Cook County can view progress on goals, like reducing
the amount of sick leave taken by government employees, using
GovStats citizen dashboard.
This chart, updated daily, shows citizens in Kansas City the percentage
of customers who have been satisfied with the citys response to water
main break service requests.
DYNAMIC REPORTING
AND COLLABORATION:
KANSAS CITY, MO
Kansas City, MOhas been a leader in
government transparency and data-
driven government. Since taking office,
Mayor James has convened regular
KCStat meetings with his senior team a nd
holds them accountable to the goals and
strategies that are most important or the
citizens o Kansas City. Te technology
behind this program began as their
homebuilt perormance measurement
platorm, KCStat. A year later, the City
added Socratas open data portal.
As the program grew, the team ound they
were limited in their reporting abilities. Sowhen Senior Perormance Analysts Kate
Bender and Julie Steenson began looking
or a perormance management solution to
enhance KCStat, they knew GovStat would
meet those needs.
O all GovStats eatures, the KCStat team
is particularly excited about the tools
drag-and-drop reporting capabilities.
We already were on board with the
concept o telling our story through data
and improving government perormance,
Bender says. Te only thing missing was
the ability to create dynamic reports and
data visualizations, she continues.
Kansas City hopes to improve how they
communicate their results to the larger
community. Te Kansas City council has
always been dedicated to civic engagement.Each monthly council meeting is filmed,
televised, and shared online, which
attracts serious stakeholders. Still, the
city wanted a way to communica
every citizen, quickly and easily.
ound the GovStat public-acing,
citizen, dashboard most useul
GovStat dashboard is a way to en
every stakeholder, says Steenson
storytelling device that makes th
more accessible.
Kansas City launched its citizen d
in early October and looks orwa
using the tool to make progress to
the citys initiatives. One big step
was having the city council adop
set o strategic priorities, which
the backbone o our da shboard,
Steenson. Te next step was assi
measurements to those prioritiescontinues. Te council made a p
statement about their priorities a
then adopted specific measures t
The use of good,
timely, accurate
data, especially
for a government
facing tough fiscal
situations, allows us
to make decisions
and prioritize at a
high level.
https://kcstat.kcmo.org/https://kcstat.kcmo.org/8/13/2019 [Socrata] Open Innovation - Volume 2
17/21
OPEN INNOVATION WINTEOPEN INNOVATION WIN TER 2014
progress. Tese metrics or tracking
progress will be the advantage o theGovStat dashboard as we build it out.
Kansas City plans to roll out a new goal
every month over the next six months,
into 2014. Residents can ollow each goal s
progress on the citizen dashboard. In the
meantime, Bender is excited to see how the
tool helps improve efficiency. Dynamic
reporting saves so much time, she says.
Its great to work in a system designed
around government use.
TAKING PERFORMANCE
MANAGEMENT TO THE NEXT
LEVEL: SAN MATEO COUNTY, CA
Te San Mateo Countygovernment
is known or its deep commitment to
transparency. Te Countys Shared
Vision 2025, a comprehensive community
planning process designed to get direct
input rom citizens, is a sterling ex ample o
this open, collaborative governing style. So
when county residents approved a hal-
cent sales tax increase in 2012, the Board
o Supervisors promised that residents
would be able to see how their tax dollars
were being spent.
SMC Perormance eatures a dashboard
with a series o interactive tiles, each tile
acting as a hub or the goals associated
with a specific Measure A initiative. By
clicking on the tile, users can explore
perormance measures, review raw data,
read the actual proposal and, in some
cases, look at charts, graphs, and maps
created with the data. One o my avorite
things about GovStat is how easy it is to
create maps and other visualizations o
the data we collect, says Shanna Collins,
a Budget Analyst in San Mateo Countys
Office o Budget and Perormance. Tese
visualizations help us identiy where the
greatest needs are throughout the county,
so we can make budget choices that are
based on objective data.
In the coming months, San Mateo County
will use GovStat to track the perormance
o the original programs unded by
Measure A to monitor progress in key
areas. In addition, the County is planning
to roll out a new dashboard to ollow the
nine community impact goals that make
up its Shared Vision 2025 in early 2014.
GovStat has been at the heart o our move
toward true data-driven management,
says Farrales. And it gives us a platorm
or involving the community and our
employees in decision-making, which is
central to our mission.
Executives in the County Managers Office
viewed the approval o the tax known
locally as Measure A as an opportunity
to take the countys transparent, data-
driven approach to budgeting to the
next level. Te Measure A sales tax will
generate about $65 million in revenue
each year over the next 10 years, says
Reyna Farrales, Deputy County Manager
or San Mateo County. We have a duty
to show how the services unded by the
tax contribute to specific, measurable
goals and how those results ultimately fit
together with the priorities in our Shared
Vision 2025.
When voters approved the sales tax
increase, county leaders began theirsearch or a technology solution to keep
the community inormed about progress
in the coming years. We were aware o
the success o the StateStat program in
Maryland and were really impressed with
the perormance dashboard they were
using, says Farrales. We discovered that
Maryland, along with a number o other
cities and states, were all using the Socrata
platorm. Our newly appointed Chie
Innovation Officer (CIO) Jon Walton came
rom San Francisco and had experience
with Socrata, so he a nd his staff were able
to help us ramp up our Socrata-powered
open data portal and move into GovStat
right away.
Afer implementing GovStat, the County
Managers Office used the system to help
departments define goals and metrics
or their respective Measure A unding
proposals. In September 2013, San
Mateo County became the first county
government in the U.S. to deploy a public-
acing GovStat site. Te launch o SMC
Perormance coincided with the Board
o Supervisors approval o 22 projects
totaling more than $50 million.
vStats mapping
pabilities help
tablish patterns.
e map above shows
water leak, water
eter, and hydrant
pair requests open in
nsas City.
This pie chart shows the number of emergency response vehicles by
category for San Mateo.
GOALS IN ACTION
Each o these organizations offer
compelling story o what is possib
by putting the principles o data-
driven decision-making, perorm
measurement, and delivery into a
Greater adoption o these practic
a trend; its a undamental shif i
way governments around the wor
embracing their mission.
n September 2013,
an Mateo County
ecame the first
ounty government
n the U.S. to deploy
public-facing
ovStat site.
https://performance.smcgov.org/https://performance.smcgov.org/8/13/2019 [Socrata] Open Innovation - Volume 2
18/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2013
CODELESCENCE:ENGINEERINGCOMES OF AGE
y Patrick Hasseries
aff Writer
ocrata is maturing, and it is most evident
hrough the evolution of its engineeringpast,
resent, and future.
Whats really exciting about Socrata right now
that were transitioning from a startup in
he true sense of the word to a company,
ays Steve Greenberg, one of Socratas newest
ngineering Program Managers.
When Socrata was
originally ounded,
it began with a small
team o scrappy
sofware engineers. eam structure
was simple, and early team members
quickly developed an office culture that
paralleled the products they were building:
open, engaging, and collaborative. Tey
encouraged ree discussion, shared a love
or challenges, and held events planned
or spontaneous that built cama raderie.
Since the earliest days o the company,
dozens o engineers have joined
Socrata, rom part-time high school
interns to veteran programmers with
decades o education and experience.Te engineering team has grown into
an engineering department, and the
original inrastructure must evolve to
accommodate more members and
bigger projects.
DIVIDING TEAMS WITHOUT
CAUSING DIVISION
When companies expand, they must be
careul not to let growth impede progress
or dull internal culture. So how will
Socratas engineering team restructure
itsel without sacrificing its liveliness?
Were emulat ing Spotiys ribes, Guilds,
Squads model, says Jerome Gagner,
Socrata Director o Engineering. Trough
it well keep the culture that we have as
well as develop engineers careers and
individual disciplines.
Based on Spotiys model or structuring
internal teams, Socratas engineers are
reorganizing into three types o groups.
All engineers dedicated to a specific
service or product orm large teams
called ribes. All engineers who share a
discipline (ront-end development, back-
end development, user interace design,
etc.) collaborate through inter-tribal
groups called Guilds. Within each tribe
are also Squads, small teams dedicated
to specific disciplines within the tribe.
Squads act like miniature startups in their
own right, maintaining their own cultures
and core values.
A CULTURE OF CODE
AND CHARACTER
Anyone who has visited Socrata can attest
to the dedication and personality that
each employee brings to the company,
particularly the engineers.
One thing I requently talk abou
riends outside o work is that So
engineering team is filled with a b
o unique, unny people. We hav
o characters here, but at the end
day, everyone is deeply committe
customers and to the companys m
says Greenberg.
Socratas engineers are inventive
rarely bored. When they arent b
running code, they run maratho
hot sauce, craf micro-brews, and
printers to produce models o Iro
helmet. In between tasks, they als
riendly euds, rom office Ner-g
to internet pranks.
radition is another important p
Socratas culture, with the engine
among its most avid ollowers. Si
companys ounding, engineers h
lunch every Friday at the same te
place in Seattles Pioneer Square.
enjoy an office happy hour on Fr
afernoons, bringing in beer or w
Data is Socratas
business, and it handles
datasets of all sizes
from those with only
a dozen records to
others with records
ranging in the millions.A search request on
a small dataset takes
only a few seconds, but
advanced searches on
large datasets naturally
take much more time
to process. Socratas
engineers have recently
enjoyed the challenge
of developing methods
to make even advanced
searches in large
datasets complete in less
than four seconds.
Challenging
QueriesSocratas engineering team
members have attended
dozens of hackathons
around the world, serving
as instructors, presenters,
and judges.
Anyone who has visited Socrata can attest to the
dedication and personality that each employee
brings to the company, particularly the engineers.
http://www.scribd.com/doc/113617905/Scaling-Agile-Spotifyhttp://www.scribd.com/doc/113617905/Scaling-Agile-Spotifyhttp://www.scribd.com/doc/113617905/Scaling-Agile-Spotifyhttp://www.scribd.com/doc/113617905/Scaling-Agile-Spotify8/13/2019 [Socrata] Open Innovation - Volume 2
19/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2014
share. On their anniversary o employment, team
embers also share one pound o chocolate or every
ar theyve worked at Socrata.
hile these perks and traditions are part o what
akes Socrata great, ask the engineers what they
ve most about their job and they will likely reply
arning, coding, and problem-solving.
We ace a lot o hard engineering challenges, and
ose challenges are very exciting to work on, says
nthony Nowell, a Socrata Sofware Developer. I
ve no doubt that Im surrounded by ridiculously
ight people who can overcome those challenges, and
m really encouraged knowing that were helping each
her to grow technically.
XPONENTIAL GROWTH
oneering tech companies like Google, Spotiy, and
cebook have demonstrated that well-defined culture
d open organizational models are valuable growth
ols. Tey help attract new tea m members, develop
mployee competency, and oster loyalty, which
turn inspires employees to produce innovative
ojects that put the company on the leading edge
entrepreneurship.
crata adds members to its team on a near-weekly
sis, many o them engineers. Part o this growth
ccess comes rom the attract iveness o Socratas
lture, which the company has nurtured since its
unding. Moving orward, Socrata will continue
emphasize organizational models that maintain
at culture.
s amazing seeing Socrata grow as big as it has, says
hris Metcal, Director o Developer Platorm and six-
ar Socrata employee. I dont have kids but, to me,
is is like seeing my child go off to college.
CHRIS CHARMS ARMSTRONG
Achievements unlocked:Helped
develop a number of startups
Geeks out on:Baking, beer, current
events, dev-ops, live music, tabletop
games, travel
Most likely to:Become a pastry chef
JEROME GAGNER
Achievements unlocked:Taught
himself to read and speak Arabic;
deployed numerous large-scale
applications in languages such as Java,
PHP, .Net, Ruby, etc.
Geeks out on:Camping, flight
simulators, SCALA, server-side and
front-end technologies
LILIA GUTNIK
Achievements unlocked:Talk
her prank adventures on The M
developed GovStat from the gr
Geeks out on:Data visualizati
Keanu Reeves, stand-up comed
Most likely to:Sleep with a K
Reeves body pillow
GIACOMO FERRARI
Achievements unlocked:3D printed
Iron Mans helmet and arc reactor; co-
created Socratas charting library
Geeks out on:3D printing, electronics,
web development
Most likely to:Invent a working flux
capacitor, blow himself up in a lab
STEVE GREENBERG
Achievements unlocked:Recently had
his first child, helped design Microsofts
error reporting system
Geeks out on:Cycling, family,
remodeling his house
KARIN HELLMAN
Achievements unlocked:Iv
it through a year in the U.S. wit
getting fat.
Geeks out on:Climbing, color
painting, Photoshop
MeetSocratasEngineers
When we say geek out, we mean
it. Socratas engineering team hosts
a bright, quirky cast of characters
who are known for much more than
creating great software.
8/13/2019 [Socrata] Open Innovation - Volume 2
20/21
OPEN INNOVATION WINTEOPEN INNOVATION WINT ER 2014
OHN KEW
chievements unlocked:Worked on
number of evolutionary computation
ojects; developed code used on
llions of servers around the world
eeks out on:Family, illustratingcroorganisms, microbiology
AYN LESLIE-COOK
Achievements unlocked:My kids
dont seem like theyll turn out to be
criminals.
Geeks out on:Family, statistical
modelling, swimming, theatre
ANTHONY NOWELL
Achievements unlocked:Recently had
his first child, built a small application to
assist his mother-in-law with her daycare
business
Geeks out on:Coding, family, webframeworks
PAUL PARADISE
Achievements unlocked:Helped
shape Socrata from the very beginning
Geeks out on:Computer hardware,
computer networking, triathlon
JEFF SCHERPELZ
Achievements unlocked:Led Socrata
through several versions of its front
end (including conversion from FLEX to
Javascript); designed and built his own
house
Geeks out on:Cars, food, knitting
ASON KROLL
chievements unlocked:Built a
stem to identify the best type of
ucation for over 15,000 careers and
d schools offering such education
eeks out on:Computer science,
onomics, music, running, statistics,
nnis
CHRIS METCALF
Achievements unlocked:Helped
Socrata grow from a startup into a
company
Geeks out on:APIs, developer tools,
home improvement, micro-brewing,
photography
BRIAN OLDFIELD
Achievements unlocked:Accepted to
grad school
Geeks out on:Bitcoin mining, websitedesign
DAN RATHBONE
Achievements unlocked:Travelled
and worked all over the world
Geeks out on:Database technologies,hiking, running, triathlon
Most likely to:Wander off to explore
and then be found living in a hut in the
middle of nowhere
CLINT TSENG
Achievements unlocked:Taught
math and English in India; worked on
a project deployed to 80+ countries
and the international space station
Geeks out on:Design, guitar, music,
movies, sailing
Most likely to:Freak that this was
written in Comic Sans
eet Socratas Engineers
8/13/2019 [Socrata] Open Innovation - Volume 2
21/21
(206) 340-8008
83 S. King St., Ste. 107
Seattle, WA 98104
www.socrata.com
twitter.com/socrata
facebook.com/socrata
https://twitter.com/socratahttps://www.facebook.com/socratahttps://www.facebook.com/socratahttps://twitter.com/socrata