8/12/2019 SQL2014 Updating Your Skills MVA Module 3

1/12

03 | Managing Data Quality

Graeme Malcolm | Data Technology Specialist, Content Master

Pete Harris | Learning Product Planner, Microsoft

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

2/12

Data Quality Services

Data Cleansing

Data Matching

Module Overview

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

3/12

The Data Quality Problem

Business decisions rely on trusted data Data quality issues in data sources can lead to inaccu

reporting and analysis Invalid data values (e.g. Californa)

Inconsistencies ( e.g. California and CA Duplicate business entities (e.g. Jim Corbin, James Corb

SQL Server DQS is a knowledge-based solution for: Data Cleansing

Data Matching

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

4/12

Knowledge Bases and Domains

Knowledge base: Repository of knowledge about data

Domain validation rules

Matching policies

Domains: Specific to a data field (or a composite field)

Contain values and validation rules for members Valid(e.g. California and CA for a Statedomain) Invalid(e.g. 90261 for a Statedomain Error(e.g. Californa for a State domain)

Define rules to correct values to leadingvalues

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

5/12

Demo: Creating a Knowledge Base

In this demonstration, you will see how to:

Create a Knowledge Base

Perform Knowledge Discovery

Perform Domain Management

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

6/12

Data Cleansing Projects

1.Select a knowledge base

2.Map data columns todomains

3.Review suggestions and

corrections4.Export results

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

7/12

Demo: Cleansing Data

In this demonstration, you will see how to:

Create a Data Cleansing Project

View Cleansed Data

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

8/12

Cleansing Data in SSIS

SSIS includes a DQS leansing Transformation

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

9/12

Data Matching

Define matching rules for business entities in a

matching policy

Rules match entities based on domains: Similarity: Similar or exact match

Weight: Percentage to apply if match succeeds

Prerequisite: Mandatory domain match for rule to succeed

If the combined weight of all matches meets or exceeds the minimum matching score, the entities are duplicates

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

10/12

Demo: Matching Data

In this demonstration, you will see how to:

Create a Matching Policy

Create a Data Matching Project

View Data Matching Results

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

11/12

Module Summary

Create a Knowledge Base for your data Use data cleansing to ensure consistent, correct dom

Use data matching to identify duplicate data entities

8/12/2019 SQL2014 Updating Your Skills MVA Module 3

12/12

2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other produc t names are or may be registered trade

U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this pre

must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of

the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.