AmFam’s Metadata Journey – What a Ride It Has Been!
WI DAMA ChapterA Focus on Data Management
12/1/2015
2
Metadata colors and informs everything we do…
it’s difficult to navigate without it!
Metadata – Helping Get us from Here to There
3
• Critical component of a complete data set and is key to preserving data’s usefulness over time.
• Quality metadata answers– what was created?– why?– when?– where?– purpose?
• Describes business context & technical characteristics, e.g., how it is– defined– calculated– managed – governed – classified
• Serves operational and long-term usage/analytical goals– facilitates the understanding of the data– used to facilitate the retrieval and storage of data, and – critical to preserving the usefulness of data
Metadata Refresher - More than just “data about data”
4
Pre-August 2013Data management de-centralized- No centralized metadata repository
- Business metadata found everywhere and nowhere- Metadata not fully understood
- Beginnings of data definition governance
Data Model Dictionaries
Operational Systems
Sharepoint Documents
Spreadsheets Access Databases
Reporting Tools
Data definitions could (or could not) exist at any of these locations, but did not always flow and inform other efforts.
AmFam’s Metadata Journey
5
August 2013Enterprise Data Management Department
Pre-August 2013
Experienced professional hired to centralize data management practices
Data Models
Operational Systems
Sharepoint
Spreadsheets Databases
Reporting Tools
Based on DAMA-DMBOK Framework, © 2008 DAMA International
Metadata & Reference Data
Data Analytics & Reporting COE
Data Governance & Data Quality
Data Program Management
Enterprise Data Strategy
Business Data Analysis
Enterprise Data Management
AmFam’s Metadata Journey
6
November 2013 - Metadata Unit Formed - Manager Hired November 2013- Staff hired Jan-Dec 2014
Metadata & Reference Data Manager - Library and Information Science Background- Previously implemented a metadata repository
Metadata and Reference Data Team
- Taxonomy/Ontology Expertise- Metadata Tool Expertise- AmFam Data (structured and
semi- unstructured) Expertise- Reference Data Expertise
Data Model Dictionaries
Operational Systems
Sharepoint
Spreadsheets Access
Databases
Reporting Tools
So - how do w
e help?
Metadata & Reference Data
Manager
EDM Director
AmFam’s Metadata Journey
7
Metadata Management
Manage metadata in a central metadata repository…
…with Data Governance checkpoints
Enterprise Data Management
Metadata & Reference Data
Business Data Analysis
Data Analytics & Reporting COE
Data Governance & Quality
Data Program Management
Enterprise Data Strategy
Manager: Lydia Reid
8
Current Stats
9
1. Facilitate Common Business Vocabularya. Work with data stewardsb. Industry standards Adopt, Adapt, Create
i. NAICii. FASBiii. IRMIiv. Etc.
2. Manage metadata quality standardsa. naming1
b. definitions2
3. Audit SDLC data requirements for a. Quality b. Consistent usage across projects
Business Metadata Management Enterprise Data Management
Metadata & Reference Data
Business Data Analysis
Data Analytics & Reporting COE
Data Governance & Quality
Data Program Management
Enterprise Data Strategy
Manager: Lydia Reid
1. More than a data dictionary or simple glossary
2. Corporate level business terms with not only definitions, but also business rules, usage context, CDE and sensitivity indicators, etc.
3. Concepts are reusable building blocks for data element definitions
1. National Information Standards Organization. “Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies.” NISO.org, accessed December, 2014, www.niso.org/apps/group_public/download.php/12591/z39-19-2005r2010.pdf.
2. International Organization for Standardization. ISO/IEC 11179-4, Second Edition 2004. 4/18/2007; Inmon, W.H., O'Neil, B.K., & Fryman, L. 2008. 4.3.3 Miscellaneous Guidelines. Business Metadata: Capturing Enterprise Knowledge. [Amsterdam]: Elsevier/Morgan Kaufmann;O’Neil, Bonnie “Business Metadata: How to Write Definitions”. Business Intelligence Network, accessed December 2014, http://ww.b-eye-network.com/view/734.
10
1. Determine if data sources are in metadata repositorya. If not, document and link to targets.
2. Review Project Charter and requirements document for concepts that have either already been defined or need to be defined for re-use in the creation of data element requirements.
3. Draft names/definitions for concepts not previously captured – include in requirements document for re-use for data element names (with appropriate class words) and definitions.
4. Review data element names and definitions created by business analysts to ensure a. reuse of previously approved terms and b. correct application of general metadata requirements for new data elements
5. Review data models to ensure standardized data requirements are re-used.
6. Ingest new project artifacts and data models into metadata repository.
7. Work with internal analytic community group to ensure contextual information is provided for/re-used, e.g., data mart explanation, semantic layer products.
Metadata - SDLC Tasks
11
Uniform Policy Retention ID: A value that provides identification of a uniform policy for the purpose of tracking retention.
Uniform Policy – new concept willrequire a new definition.“Uniform Policy” will exist as a Narrower Term of “Policy”.
ID – Data Element Class Word; all IDs or identifiers must begin with “A value that provides unique identification of…”
Policy – Data Governance approved definition:A combination of documents that makes up the insurance contract, consisting of…
No need to repeat the policy definition in this entry.
We typically try to avoid specific timevalues within the definition. We try tocapture this information through thebusiness rules so that the conceptual Definition stays valid even when therules change.
Retention – Retention is an approved term. Does this “Retention” refer to the same thing?If there is a difference, is this a narrower version of Retention?New term recommended: Uniform Policy Retention.
Example of the type of analysis the Metadata team does for SDLC data requirements.• This particular term and definition are well-written within the context of a particular project.• The Metadata team tries to identify how the data element will be able to exist outside of the confines of the project in the future.
Common Business Vocabulary - SDLC Analysis
12
Common Business Vocabulary - SDLC
Reuse, reuse, reuse – or create new according to standards
Do not re-define concepts
1. Train Business Analysts
2. Retire divisional glossaries
3. Create specialized search templates in metadata repository
4. Provide Simple Search API
5. Provide “Accelerator” or “Hot Keys”
13
IE Accelerator plugin
Internet Explorer (IE) Accelerator:a. On a web page highlight a termb. Right-clickc. Select CBV accelerator
14
Advocacy!
15
But does it really work?
Only if you create quality navigational pointers….and
LEVERAGE THEM!
16
Map of the Big Dig (Central Artery / Tunnel Project) Big Dig Under Construction
Leveraging Metadata to Find Your Way
Map the way before the travel routes are built or improved.
17
Avoid overbuilding
The Inner Loop … It’s underused, overbuilt…*
*Mayor Thomas S. Richards Speech at the Rochester Downtown Development Corporation's Dec. 6, 2011 Vision Future Event; http://www.cityofrochester.gov/article.aspx?id=8589951197 accessed 9/8/2014
Leveraging Metadata to Find Your Way
18
Leveraging Metadata to Find Your Way
Arrive at the same destination
To
Or
19
Take the Safest Route
Leveraging Metadata to Find Your Way
20
Avoid heavy tolls
Leveraging Metadata to Find Your Way
21
Manage Access
Leveraging Metadata to Find Your Way
22
Allow for the Scenic Route
Leveraging Metadata to Find Your Way
23
Who Benefits?
1. Common Business Vocabulary = data context (All AmFam)
2. Data Ownership (All AmFam)
3. Data discovery and data context (Data Analysts/Scientists)
4. Impact Analysis (BI Solutions)
5. Identification of duplicate solutions (I/S Architecture)
6. Access to CDE data profiling (Data Owners)
7. Inventory of sensitive data (I/S Security/Customers/Employees)
Sample Benefits (Primary Beneficiaries)EVERYONE!
Which means everyone can make better decisions with well-managed, high quality metadata.
24
The Journey Continues
• Metadata Management is Not - A one-and-done type deployment
• Metadata Management Requires- Early attention and maintenance throughout the data lifecycle- Quality standards and audits as part of SDLC
• Metadata now incorporated into
Information Standards and Governance
25
Contact Information
Lydia J. E. ReidInformation Standards & Governance
Enterprise Information Management (EIM)[email protected] | 608.242.4100 ext. 32591
Thank you for your attention, and I wish you Godspeed
on your own Metadata Journey