Creating the Foundation for Enterprise Information Management

Creating the Foundation for Enterprise Information Management

+The Problem

+Global IDs Focus

+Software Functionality

+Automation

+Enterprise-Level Scalability

+Global IDs Value Offering

About Global

IDs

3

Organizations have growing cost & complexity

Most large organizations have many thousands of databases that drive up complexity and cost.

Complex data ecosystems

Highly dynamic

Limited traceability

Systemic Risk : Hard to measure

About Global IDs

section 01Global IDs ©2015

DataLandscapes

4

Global IDs FocusAbout Global IDs

To help large organizations reduce the cost of managing enterprise information by using software that is

• Systematic• Automated• Scalable

and can manage complexity in enterprise data landscapes.

section 01

Global IDs ©2015

5

Software FunctionalityAbout Global IDs

section 01

Global IDs ©2015

1. Scan

2. Analyze

3. Map / Organize

4. Govern

6

AutomationAbout Global IDs

section 01

Global IDs ©2015

7

Enterprise-Level ScalabilityAbout Global IDs

section 01

Global IDs ©2015

Metadata Discovery Tasks

Data Quality Tasks

Data Profiling Tasks

Data Mapping Tasks

Data Monitoring Tasks

Massively parallel distributed computing

8

Global IDs Value OfferingAbout Global IDs

section 01

Global IDs ©2015

Improve TransparencyUsing Metadata Governance

Improve QualityUsing Data Quality Audits

Improve TraceabilityUsing Data Lineage Analysis

Increase Trust in DataMaster Data Governance

Increase Operational Efficiency

Reduce Costs Increase Revenues

Identify RedundancyGraph Analysis of Duplicate Data

Support AutomationIdentify Manual Steps for Automation

Run-time Data Validation & Cleansing Accelerates Data Cleansing

Upwards Of 500%

Customer 360 AnalysisUsing Linked Data

Customer-Product Predictive Analysis

Using Graph Databases

+ Introduction

+Why download Global IDs trial server

+ How to get a trial server

Trying out Global IDs on your own

trial server

10

Introduction

Try out Global IDs software on Amazon Web Services platform without downloading it. You can now get a hands-on experience on the product suite on the cloud.

The portal environment lets you scan and discover databases, profile data in them, recognize and organize information and map them appropriately. Global IDs provides a set of useful workflows that simplify your work.

Try out Global IDs software independently without downloading it.

Trying out Global IDs on your own trial server

section 02Global IDs ©2015

11Global IDs ©2015


Why try out Global IDs server

section 02

Understand your data better by running it through standard data management processes that the trial server offers

Learn Data Management and Governance using Global IDs

Get certified on Global IDs platform

Assess Global IDs for using it in your organization

12

You can request the trial server from our website (www.globalids.com). You need to perform the following steps.

• Use the following link - http://www.globalids.com/request-global-ids-server

• The Request Global IDs Server form appears. You need to fill up the form and click Submit button.

• An email gets sent to you with details about the trial server. This takes 5 -10 minutes.

• Click on the link for accessing the trial server (in the email) to browse the Data Transparency Portal.


Form for getting access to your trial server

How to get a trial server

Global IDs ©2015

section 02

http://www.globalids.com/request-global-ids-server

+ Add connections

+ Run workflows

+ Inspect metadata and profiling results

+ Outliers

+ Organize

+ Map

What to do after you get a

trial server

14Global IDs ©2015

What to do after you get a trial server

Working with connections

section 03

Notice two connections already added

Add connections in bulk from an Excel sheet

Add connections manually

15

Working with already added connections


For the two connections already added, you can run any of the following workflows.

Discovery, Profiling and Recognition Workflow - This workflow lets you discover, profile, organize and map tables.

Discovery Workflow - This workflow tables, their metadata and explicit relationships.

Global IDs ©2015

section 03

16

• Click Discovery menu Connections section Actions drop-down menu Add Connection.

• You can connect to the PRODUCT_HUB database on the trial server.

• In the wizard, provide:

Connection name

System name - select the trial server name

Instance name - PRODUCT_HUB

Database type - Derby

Port - 1530

Username - sample & password - sample. Click Test Connection to test the connection

Database Owner’s name, Database Administrator’s name & Data Steward’s name

Database category & Business entity

Password expiration date


Add connections manuallyThe added connection shows up in the tabular view on the Connections section under the Discover menu when the table is refreshed. After the connection is added, the Discovery, Profiling and Recognition Workflow can be run to profile, organize and map the tables.

Global IDs ©2015

section 03

17

• Click Discovery menu Connections section Actions drop-down menu Add Connections in Bulk.

• In the wizard:

The connection file has to be selected from the server system or local system. A sample file is selected by default. Use this file.

Database User - sample & Database Password - sample.

The connections listed in the Excel file are created and are displayed on Discover menu Connections section when the table is refreshed.

The Discovery, Profiling and Recognition Workflow can be run on each of the added connections to scan, profile, classify and organize the data landscape.


Add connections in bulk from an Excel sheet

Global IDs ©2015

section 03

18

Run Workflows

Global IDs ©2015


Menu Workflow Purpose

DiscoverAdd Connection and Create Transparency Workflow

This workflow connects to a database instance and automatically starts the discovery, profiling, classification and recognition processes.

DiscoverDiscovery, Profiling and Recognition Workflow

This workflow discovers and profiles database instances and also runs the classification and recognition process.

DiscoverDiscovery Workflow This workflow discovers tables in a database, discovers

their metadata and documented relationships.

DiscoverAdd and Profile File Workflow

This workflow scans delimited and Excel files, discovers their metadata, loads their data in the database and then profiles them.

ProfileRun Profile Relationship Workflow

This workflow finds documented, undocumented and user-defined relationships between tables in databases.

MapFind Database Clones Workflow

This workflow finds similar database instances in the landscape that have a 95% match.

section 03

19


Add Connection and Create Transparency Workflow

Global IDs ©2015

The Add Connection and Create Transparency Workflow performs the following activities:

Creates connection to database

Discovers tables, metadata & explicit relationships

Generates Metadata Report

Profiles columns, relationships & domains

Generates Profile Report

Classifies and organizes tables

section 03

20

Click Discover menu Connections section Actions drop-down menu Add Connection and Create Transparency Workflow.

• In the wizard:

select the connection name.

System name – select the trial server name.

Instance name – PRODUCT_HUB

Database Type – Derby

Port - 1530

Username - sample & password - sample. Click Test Connection to test the connection.



Add Connection and Create Transparency Workflow (cont.)

Global IDs ©2015



section 03

21


Discovery, Profiling and Recognition Workflow

Global IDs ©2015

The Discovery, Profiling and Recognition Workflow performs the following activities:

Discovers tables, metadata & explicit relationships


Profiles columns, relationships & domains

Generates Profile Report

Classifies and organizes tables

Pre-requisites:

Connection to database

section 03

22

Click Discover menu Metadata section Actions drop-down menu Discovery,Profiling and Recognition Workflow.

• In the wizard:


select the schemas on which you want to run the workflow.

provide the user credentials. For the two added connections the credentials are provided by default.

Check the appropriate checkboxes for discovering tables, views and synonymous tables & for finding row count of tables from system tables.


Discovery, Profiling and Recognition Workflow (cont.)

Global IDs ©2015

section 03

23


Discovery Workflow

Global IDs ©2015

The Discovery Workflow performs the following activities:

Discovers tables in databases

Scans metadata of discovered tables

Discovers documented/explicit relationships


Pre-requisites:

Connection to database

section 03

24

Click Discover menu Connections section Actions drop-down menu Add Connection and Create Transparency Workflow.

• In the wizard:


System name – select the trial server name.

Instance name – PRODUCT_HUB

Database Type – Derby

Port - 1530

Username - sample & password - sample. Click Test Connection to test the connection.



Discovery Workflow (cont.)

Global IDs ©2015



section 03

25

Track Workflow Progress


Click Account menu My Actions sub-menu

You can view:

Running Tasks - This screen lists the currently running task groups submitted by the workflow.

Running Workflows - This screen shows the currently running workflow.

Running Tasks

Running Workflows

Global IDs ©2015

section 03

26

Track Workflow Progress (cont.)


Click a running task group from the Running Tasks screen.

You can view:

Running Tasks - This screen lists the currently running tasks in the task group.

Summary - This screen shows a summary of the tasks that are running or have finished

All Tasks – This screen lists all the tasks along with their status

Scheduled Tasks – This screen shows the tasks that have been scheduled

Paused Tasks – This screen shows the tasks that have been paused

Failed Tasks – This screen lists the tasks that have failed

History – This screen displays history of all the tasks in the workflow

Global IDs ©2015

section 03

27

Inspect Metadata and Profiling Results

Results of Data Discovery:

Click Discover menu Metadata section.

This screen lists all the schemas whose metadata has been discovered. Click on a schema to view the following:

Tables - Lists all tables in the schema along with their metadata

Relationships - Shows the various documented relationships

between tables in the schema

Table Descriptions – Displays the documented descriptions of tables

SQL Editor – This screen allows running SQL queries on the

discovered tables

Drill-down is available for viewing metadata of columns

Metadata

Data

Relationships

Indexes

Descriptions


Global IDs ©2015

section 03

28

Inspect Metadata and Profiling Results (cont.)

Results of Column Profiling:

Click Profile menu Profile section.

This screen lists all the schemas which have been column profiled. Click on a schema to view the following:

Profile Summary – Displays a summary of the profiling results for all

tables in the schema whose metadata has been discovered

Outliers – Lists various outliers detected while profiling

Highlighted Outliers – Displays all outliers that have been

highlighted

Metadata– Displays metadata of columns of tables that have been

profiled

Indexes –Lists indexes in the schema

Relationships – Lists the various Primary Key – Foreign Key

relationships between tables in the schema (both documented and

undocumented/inferred)

Table Descriptions – Displays documented descriptions of tables

SQL Editor - This screen allows running SQL queries on the tables

Drill-down is available for viewing profiling results of columns

Profile

Outliers

Highlighted Outliers

Descriptions

Profile History


Global IDs ©2015

section 03

29

Inspect Metadata and Profiling Results (cont.)

Results of Relationship Profiling:

Click Profile menu Relationships section.

This screen lists all the schemas which have been profiled. Click on a schema to view the following:

Documented Relations - Lists all documented/explicit Primary Key –

Foreign Key relationships between tables

Inferred Relations – Lists all Primary Key – Foreign Key relationships

between tables that have been detected through relationship

profiling

User Defined Relations –. Displays all user-defined relationships

between tables . Also allows creation of Primary Key – Foreign Key

relationships

Relationship Analysis

Cardinality (Average,

Maximum & Minimum)

Orphan Values

Childless Values


Global IDs ©2015

section 03

30

Reports


Metadata Report

Domain Report

Profile Report

Discover Reports

ProfileReports

OrganizeDomain Reports

Global IDs ©2015

section 03

31

Outliers


Global IDs ©2015

Character Distribution

Outlier

Datatype Outlier

Frequency Outlier

Length Outlier

Pattern Outlier

Value Outlier

The software, post profiling activity, highlights/suggests certain values as outliers or possible errors.

Click Profile menu Outliers section

Values with the least occurring data type (column

level) get marked as

outliers

Values with improper/unu

sual character

distribution (column

level) get marked as

outliers

Values occurring

once where other values

in the column occur

multiple times and vice versa

get marked as outliers

Values of the least

occurring length

distribution in a column get

marked as outliers

Values with the least occurring

Regex pattern in a column get marked as

outliers

Calculated on the basis of Six Sigma

Methodology. Values

outside the +6 and -6

sigma range get marked as outliers

section 03

32

Domains and Semantic Objects

Domains are business specific concepts represented in the data landscape. They are associated with reference values or patterns.

Types of Domains:

Global

Business

Auto Generated

View domains from Organize menu Recognition Domains

The product organizes the domains into groupings called semantic objects.

View semantic objects from Map menu Semantic Objects


Global IDs ©2015

ID First Name

Last NamePhone

Number

City Country

CUSTOMER

Semantic Object

Domains

section 03

33

The software automatically creates a “bucket” for each domain and places each column that it recognizes into the appropriate domain.

• Click Organize menu Recognize section Domain Profiling section to view the total number of columns domain profiled in the schema.

• Click the schema name to view:

Domain Profile Status – Shows summary of profiling activity

Domains – Lists tables in the schema and shows the details of columns recognized during the domain profiling activity

Domains (table level) – Shows domain profiling details for columns of a table


Interpret results of domain recognition

Global IDs ©2015

Domains Tab

Domains Tab (table level)

section 03

34

The software automatically locates similarities between semantic objects and tables. These similar tables get classified with objects under 4 classification heads –

Location

Product

People

Organization

Similarities are detected in two ways:

Domain based – based on data similarity between domains of objects and columns of tables

Name based – based on similarity between domain and column names


Interpret results of object classification

Global IDs ©2015

section 03

35

Results can be reviewed in two ways:

Map menu Semantic Mappings section

Map menu Semantic Objects section

click a semantic object from the list

Mapping Details section


Review results of automatic domain recognition

Global IDs ©2015

Mapping Details

section 03

36

Click Map menu Semantic Objects section Actions drop-down menu Add Object.

• In the wizard, provide the following:

Scope – select Global or Business

Name – name of the semantic object.

Logical name

Description

Notes

URL

Domains – select the domains that would be included in the semantic object


Create Semantic Objects

Global IDs ©2015

The added semantic object is listed under Semantic Objects section when the tabular view is refreshed.

section 03

37

You can map domains of objects to columns of tables :

manually

from maps suggested by the software

Click Map menu Semantic Objects section click a semantic object from the list Suggested Mappings section.

Select a row on the top

Suggestions tab at bottom shows the various domains of the object on the left. On the right, drop-down lists are provided for selecting appropriate columns of the selected table.

Notice that in some drop-down lists columns are already selected. Those are the mappings suggested by the software through domain recognition activity.

Check the checkboxes next to all the columns you want to map. Select the appropriate columns from the drop-down lists. Click Save Mappings button.

The mappings are listed in the Mapping Details section.


Map Semantic Objects

Global IDs ©2015

Manually:

Objects can also be mapped from Mapping Details section Map button.

section 03

Contact InformationFor more info, please contact us at

[email protected]@globalids.com609-683-1066

Thank you

mailto:[email protected]



Documents

Creating the Foundation for Enterprise Information Management