Shared Data Services in the Public Sector

WHITEPAPER

Shared Data Services in the Public Sector 2

BACKGROUND

Across the landscape of public sector agencies there’s a major ongoing paradigm shift focused on leveraging data for advanced analytics as keys to their strategic modernization efforts. Many are setting up agency-wide, shared service data environments and moving to “Agile DataOps” operating models to empower users across the organization with timely access to data, leveraging cloud native technologies where possible. This approach is aimed at leveraging shared-service models to create economies of scale for a wide range of users supported by these platforms, enabling all kinds of data-focused use cases ranging from improving business intelligence to operationalizing AI for key mission needs.

For people that have been involved in “Big Data” projects for the past decade this shift may not seem like that big of a change as it could easily be interpreted that these efforts are “just moving their on-prem data lakes to the cloud.” In reality, there are key nuances in the intent and the execution of these programs from the “data lake” projects that have long existed, and it’s not just “moving to the cloud.”

The intent of these efforts is to empower users with access to data for their specific needs. Functionally, it’s about “democratizing” data to enable users of varying skills and personas to operationalize data for their specific needs in self-service models that remove the historical bottlenecks in data pipelines, resulting in modern, mission-led data operations that are agile, scalable and end user-focused (Agile DataOps). Ultimately, it’s about empowering business users in addition to technical users. This is a major change from legacy IT-lead data operations where end users had to rely on technical resources to extract, transform and load (ETL) data for their own analysis and decision making.

This change is akin to the transformation that began in application development organizations almost a decade ago in their move to Agile DevOps operational models. In that shift, the intent was to empower developers to easily update and deploy code while removing historical bottlenecks of legacy SDLC models (anyone remember Waterfall?).

Effectively, the pattern of modernization is the same - enable the organizational modernization by removing operational barriers to empower users to “serve themselves” for their data needs, reducing overhead and expediting data-driven modernization efforts.

Shared Data Services in the Public Sector 3

The Evolution to SuccessOften driven by public policy (the DATA Act, Cloud Smart, National Defense Strategy etc.), public sector agencies are actively working to build and deliver shared service programs focused on leveraging data as a strategic asset. Many are leveraging cloud infrastructure to handle the scale while providing flexible tools to meet unique data and analytics requirements for users of varying personas across the organization.

The agencies finding the greatest success in this endeavor have designed their platforms with a balance on both empowering users to rapidly operationalize data, while adhering to governance, policy and security standards. Functionally, this is achieved by building to empower users through self-service by providing specific best-of-breed tools for data preparation, cataloging, governance and analysis to maximizing the impact across the agency. And while this may seem obvious to some, the reality is that there has been an evolution for many of these programs with lessons learned along the way.

Figure 1. Self-service data preparation combines visual guidance with intelligent suggestions to make data preparation tasks easy — no code needed

Shared Data Services in the Public Sector 4

ADVANAA great example of the evolution to success for shared data services is the Office of the Secretary of Defense’s ADVANA platform. This program was originally set up as the shared service data and analytics platform to enable the Department of Defense to deliver the congressionally mandated financial audit. Initially conceived as a “data lake,” the platform has evolved into a successful shared service to power data analysis for a myriad of DoD use cases ranging from financial reporting to force readiness.

This success was not easy and didn’t come right out of the gate. The government, aided by their systems integrator Booz Allen Hamilton worked tirelessly over a few years to evolve the platform to what it is today. The big shift really occurred when they implemented self-service capabilities for key functional areas including data prep (Trifacta), visualization and reporting. This enabled “less technical” users to operationalize data for their own mission needs quickly and easily, expanding the target user base beyond technical data professionals (data engineers/data scientists) by eliminating the need to write code or use highly technical tools to work with the data. In effect, this approach broadened the “Total Addressable Market” for ADVANA significantly and created momentum by opening functionality desired by business users. This spike in demand also drove ADVANA to outgrow their on-premises data center and move to AWS’ Govcloud.

Today, ADVANA has thousands of users from dozens of programs leveraging the shared environment for their specific needs ranging from business intelligence and reporting for decision support to enabling data scientists to test and build AI for various mission needs. In April 2020, ADVANA was chosen by the DoD to handle the COVID-19 response due to the depth of capability provided by the platform.

For More on ADVANAVisit the Booz Allen Hamilton website

Figure 2. Operationalize data workflows with ease

Shared Data Services in the Public Sector 5

MD THINKAnother example is the State of Maryland’s THINK platform. MD THINK, or the Maryland Total Human Services Integrated Network is a $200-million modernization initiative driven by the Governor’s office aimed at improving access to human services by enabling data-sharing between agencies.

Initially, the program focused on immediate needs for the MD Department of Human Services, MD Department of Health, MD Department of Juvenile Services, and the MD Health Benefit Exchange.

Due to the early phase successes, in 2019 the platform was opened up to serve any agency in the State of Maryland. Just like ADVANA, MD THINK has found success by focusing on enabling users with a self-service, cloud-focused approach. As stated on their website, “using cloud-based technologies is cost-efficient because it eliminates the need for creating these services for each individual application,” resulting in greater return on investment for the State and better service delivery to citizens of Maryland.

For More on MD THINKwww.dhs.maryland.gov/mdthink | Read the AWS Case Study

Key Takeaways from Leading Shared Data services ProgramsSimilar shared-service data and analytics programs are being launched across various Federal and State departments and agencies including the US Army, US Air Force, TRANSCOM, DHS, CDC, USAID and the State Department. There are many commonalities in the impetus for these technology efforts including establishing Chief Data Officer (CDO) data-strategy aligned shared services capabilities to support the missions of these organizations. Often this equates to leveraging “cloud native” tooling that provides the accessibility, security and scale required for enterprise data analytics needs as called out in the Federal Data Strategy Action Plan “The Federal Data Strategy provides a common set of data principles and best practices in implementing data innovations that drive more value for the public.”

As of Spring 2020, many of these are in early project phases, working through design, architecture and prototyping efforts with an eye towards scaling out production environments in the coming months. While delivering a “shared data platform” may be the “North Star” guiding these projects, the reality is that success can only be achieved when the platform is being used to deliver mission impact to a broad and diverse user base.

In looking at the evolution in the journeys of other rollouts of shared service data platforms for public sector agencies, the commonality found in those most successful is in the focus on enabling users through self-service. Agencies early in their journey to delivering a shared data platform should look at the evolution of the most successful platforms in the public sector and “skip ahead” by employing the architectural and operating models of those that have gone before. Fundamentally, this means architecting for a cloud-native, self-service driven operating model from the beginning of the project.

Shared Data Services in the Public Sector 6

This means enabling users to explore, transform and process available data to operationalize it effectively for a variety of needs ranging from building AI to improving BI insights. To achieve this, many of these leaders have found that leveraging Trifacta’s leading self-service data preparation platform has been the key component in their platforms to solve the myriad of data needs in a scalable, secure and collaborative fashion built for the cloud.

Trifacta provides a solution that meets the needs of various stakeholders with varying skill sets across the target user community, setting up a modern DataOps architecture that can handle any text-oriented data (structured to unstructured) at scale. Ultimately for programs like ADVANA and MD THINK, Trifacta’s capability allows the less technical users to automate data prep without having to write code, making them effective; while in parallel enabling more technical users (data scientists and engineers) in the user community to focus on higher value work by offloading and automating the janitorial data prep work consuming most of their time.

What this means for these shared services organizations is that they can attract more users by having a tool in Trifacta that allows business users and “citizen data scientists” to effectively work with data, expanding the total addressable market for their data platform. As a case in point, for ADVANA, this model powered in large part by Trifacta’s self-service data prep platform has enabled the program to attract thousands of users across the entire DoD with a wide range of use cases, ensuring rapid mission impact at scale. As a shared-service data platform it’s a model for success any public sector agency would relish to achieve.

Sources

https://comptroller.defense.gov/Portals/45/Documents/defbudget/FY2017/budget_justification/pdfs/2017MarchAmended/03_RDT_and_E/OSD_FY17_RDTE_ABS_20170314.pdf

https://en.wikipedia.org/wiki/Digital_Accountability_and_Transparency_Act_of_2014

https://strategy.data.gov/action-plan/

https://cloud.cio.gov/

https://www.marylandhbe.com/wp-content/uploads/2018/11/MDThink_Board_111918.pdf

“It’s impossible to overstress this: 80% of the work in any data project is in cleaning the data.DJ PatilFormer Chief Data Scientist of the United States

Shared Data Services in the Public Sector 7

About Trifacta

Trifacta is the global leader in data preparation. Trifacta leverages decades of innovative research in human-computer interaction, scalable data management and machine learning to make the process of preparing data faster and more intuitive. Around the globe, tens of thousands of users at more than 10,000 companies, including leading brands like Deutsche Boerse, Google, Kaiser Permanente, New York Life and PepsiCo, are unlocking the potential of their data with Trifacta’s market-leading data preparation solutions. Learn more at trifacta.com.

For Additional Questions, Contact Trifactawww.trifacta.com | 844.332.2821

Experience the Power of Data Wrangling Todaywww.trifacta.com/start-wrangling