Workshop Agenda
• Importance of Data Quality• What is ArcGIS Data Reviewer?• Automated review• Semi-automated review• Managing errors and reporting data quality• Summary/Resources
ArcGIS Data Reviewer: An Introduction
Defining qualityA business perspective
• Executive- Confidently make decisions- Reduce financial risk- Optimize organizational performance
ArcGIS Data Reviewer: An Introduction
Defining qualityA business perspective
• Executive- Confidently make decisions- Reduce financial risk- Optimize organizational performance
• Manager- Effective data stewardship- Drive increased usage- Maximize productivity
ArcGIS Data Reviewer: An Introduction
Defining qualityA business perspective
• Executive- Confidently make decisions- Reduce financial risk- Optimize organizational performance
• Manager- Effective data stewardship- Drive increased usage- Maximize productivity
• Knowledge worker- Increased efficiencies- Confidence in GIS
ArcGIS Data Reviewer: An Introduction
Defining QualityA Technical Perspective
Spatial Accuracy
Thematic Accuracy
Completeness
Logical Consistency
Temporal Quality
Usability
ISO-19157:2013 Geographic information – Data quality
ArcGIS Data Reviewer: An Introduction
Defining QualitySources of data quality requirements
Industry standards /
Specifications
ArcGIS Data Reviewer: An Introduction
Defining QualitySources of data quality requirements
Industry standards /
Specifications
Subject matter experts
ArcGIS Data Reviewer: An Introduction
Defining QualitySources of data quality requirements
Industry standards /
Specifications
Subject matter experts
Training and experience
ArcGIS Data Reviewer: An Introduction
Defining QualitySources of data quality requirements
Industry standards /
Specifications
Subject matter experts
Training and experience
Quality assurance
plans
ArcGIS Data Reviewer: An Introduction
Data quality managementCapabilities of the ArcGIS Platform
Geodatabase integrity
• Schema constraints• Geoprocessing tools• Data load checks• Versioning
Advanced data types
• Topologies• Geometric networks
ArcGISData Reviewer
• Automated review
• Semi-automated review
• Error management
• Quality reporting
ArcGIS Data Reviewer: An Introduction
What is ArcGIS Data Reviewer?Data Quality Management for ArcGIS
• Provides- Rule-based validation- Interactive tools- Track errors
• For individuals and enterprise- Saves time/money- Less rework
• For multiple domains- Configurable- Extendable
Validate
ReviewSummarize
ArcGIS Data Reviewer: An Introduction
What is ArcGIS Data Reviewer?Data quality management in the ArcGIS platform
• Data Reviewer for Desktop- ArcMap- ArcGIS Pro
• Data Reviewer for Server- Standard or higher
• Data Reviewer API- JavaScript- FLEX
• Web AppBuilder for ArcGIS
Desktop Web Device
Server
Online Content and Services
ArcGIS OnlinePortal
ArcGIS Data Reviewer: An Introduction
Managing Quality ControlQuality Control Processes
ReviewerResults
AutomatedReview
QualityReporting
Semi-automatedReview
ArcGIS Data Reviewer: An Introduction
Managing Quality ControlQuality Control Processes
ReviewerResults
AutomatedReview
QualityReporting
Semi-automatedReview
ArcGIS Data Reviewer: An Introduction
Types of quality control
Fast
Consistent and repeatable
Objective
100% coverage
Automatedreview
ArcGIS Data Reviewer: An Introduction
• Implementing quality requirements- 40+ configurable checks
- Feature integrity- Collection rules
- Attribute- Feature and table values
- Spatial- Spatial relationships
Automating Data Validation
http://j.mp/DRCheckPoster104
ArcGIS Data Reviewer: An Introduction
Demo Scenario
• Department plans to use temporary employees with field data collection devices to map road-related features (signs, street furniture).
• Success CriteriaFeatures should be positionally accurate based on siting
criteriaKey attributes are populated and have the correct valuesAll features should be collected within the survey areaStreamline data collection processesAll errors are corrected and verified
ArcGIS Data Reviewer: An Introduction
Batch ValidationImplementing Cumulative Review
• Reviewer Batch Jobs- Define and encapsulate quality requirements- Captures domain knowledge- Designed once and executed many times
Industry standards /
Specifications
Subject matter experts
Training and experience
Quality assurance
plans
ArcGIS Data Reviewer: An Introduction
Where to startLeveraging templates
• Data Reviewer templates - Electrical utilities- Gas utilities- Local government- Water resources- Water utilities
• Based on Esri industry models• Use as starting point
solutions.arcgis.com
ArcGIS Data Reviewer: An Introduction
Authoring Batch JobsTips and Tricks
Create sample data for testing
Configure and execute from Reviewer Toolbar
Leverage Mini-Browser to assess results
Validate a pilot area with known issues
ArcGIS Data Reviewer: An Introduction
Execute data validation using• ArcMap
Methods for executing data validation
ArcGIS Data Reviewer: An Introduction
Execute data validation using• ArcMap• ArcGIS Pro
Methods for executing data validation
ArcGIS Data Reviewer: An Introduction
Execute data validation using• ArcMap• ArcGIS Pro• Model/Python script
Methods for executing data validation
ArcGIS Data Reviewer: An Introduction
Execute data validation using• ArcMap• ArcGIS Pro• Model/Python script• ArcGIS Workflow Manager
Methods for executing data validation
ArcGIS Data Reviewer: An Introduction
Execute data validation using• ArcMap• ArcGIS Pro• Model/Python script• ArcGIS Workflow Manager• ArcGIS for Server
Methods for executing data validation
http://bit.ly/DR_BVM
ArcGIS Data Reviewer: An Introduction
Demo Scenario
• Department plans to use temporary employees with field data collection devices to map road-related features (signs, street furniture).
• Success CriteriaFeatures should be positionally accurate based on siting
criteriaKey attributes are populated and have the correct valuesAll features should be collected within the survey areaStreamline data collection processesAll errors are corrected and verified
ArcGIS Data Reviewer: An Introduction
Batch validationEvents when validation may be required
Before data loading
During editing
Prior to sharing/publishing maps
Prior to rolling-out new applications
ArcGIS Data Reviewer: An Introduction
Managing Quality ControlQuality Control Processes
ReviewerResults
AutomatedReview
QualityReporting
Semi-automatedReview
ArcGIS Data Reviewer: An Introduction
Types of quality control
Fast
Consistent and repeatable
Objective
100% coverage
Automatedreview
Semi-automatedreview
Guided Workflows
Streamlined processes
Subjective
Sampling
ArcGIS Data Reviewer: An Introduction
Semi-automated review methods
Visual review
• Redlining
• Systematic review
• Sampling
Data inspection
• Feature counts
• Attribute review
• Version differences
Positional accuracy
• Assessment tool
ArcGIS Data Reviewer: An Introduction
Value of Performing Visual Review
Discover Patterns
Compare to trusted sources
Find missing features
ArcGIS Data Reviewer: An Introduction
Semi-automated reviewLeveraging ArcGIS for Desktop
Tools supporting- Selecting/browsing features- Redlining missing features- Flagging existing features in error- Generating random samples- Assessing positional accuracy- Comparing geodatabase versions
ArcGIS Data Reviewer: An Introduction
Semi-automated reviewLeveraging ArcGIS for Server
• Extending quality control workflowsinto other communities
- QC review across ArcGIS platform
- Simple to use tools for error identification
- Manual QC workflow “automation”
ArcGIS Data Reviewer: An Introduction
Demo Scenario
• Department plans to use temporary employees with field data collection devices to map road-related features (signs, street furniture).
• GoalsFeatures should be positionally accurate based on siting
criteriaKey attributes are populated and have the correct valuesAll features should be collected within the survey areaStreamline data collection processesAll errors are corrected and verified
ArcGIS Data Reviewer: An Introduction
Managing Quality ControlQuality Control Processes
ReviewerResults
AutomatedReview
QualityReporting
Semi-automatedReview
ArcGIS Data Reviewer: An Introduction
Data Quality ReportingArcGIS for Desktop
• Adhoc summary tools- Result statistics
• Automated reporting- Automated Check Results- Total Record Count- Sampling
ArcGIS Data Reviewer: An Introduction
Data Quality ReportingArcGIS for Server
Enabling transparency in data quality- Better decision making by communicating
data quality across stakeholders- Open quality reporting- Shared across ArcGIS system- Tools and methods to communicate quality
ArcGIS Data Reviewer: An Introduction
Demo:Reporting Quality
ArcGIS Data Reviewer: An Introduction
Demo Scenario
• Department plans to use temporary employees with field data collection devices to map road-related features (signs, street furniture).
• GoalsFeatures should be positionally accurate based on siting
criteriaKey attributes are populated and have the correct valuesAll features should be collected within the survey areaStreamline data collection processesAll errors are corrected and verified
ArcGIS Data Reviewer: An Introduction
Managing Quality ControlQC lifecycle management
REVIEW
Find &Record Errors
CORRECT
Perform Editsor
Note Exceptions
VERIFY
Acceptableor
Unacceptable
ArcGIS Data Reviewer: An Introduction
Demo Scenario
• Department plans to use temporary employees with field data collection devices to map road-related features (signs, street furniture).
• GoalsFeatures should be positionally accurate based on siting
criteriaKey attributes are populated and have the correct valuesAll features should be collected within the survey areaStreamline data collection processesAll errors are corrected and verified
ArcGIS Data Reviewer: An Introduction
Workshop Review
• Importance of data quality
• Forms of data quality control- Automated review- Semi-automated review
• ArcGIS Data Reviewer- Automated validation- Semi-automated tools- Error lifecycle management- Reporting
ArcGIS Data Reviewer: An Introduction
Day and Description Type Time Location
Tuesday June 28
ArcGIS Data Reviewer: An Introduction Technical Workshop 8:30 – 9:45am Room 15B
ArcGIS Data Reviewer: Assessing Positional Accuracy Demo Theater 10:30 – 11:15am Demo Theater 4Geodata
ArcGIS Data Reviewer: Integrating ArcGIS Data Reviewer and ArcGIS Workflow Manager to Automate Data Quality Control Workflows
Demo Theater 11:30 – 12:15pm Demo Theater 4Geodata
ArcGIS Data Reviewer: Leveraging Geoprocessing for Data Validation Demo Theater 12:30 – 1:15pm Demo Theater 4
Geodata
ArcGIS Data Reviewer: Quality Assessment for Elevation Raster Datasets Demo Theater 2:30 – 3:15pm Demo Theater 4
Geodata
Wednesday June 29
ArcGIS Data Reviewer: An Introduction Technical Workshop 3:15 – 4:30pm Sapphire Ballroom
A/B
Thursday June 30
ArcGIS Data Reviewer: Advanced Data Validation Concepts Technical Workshop 8:30 – 9:45am Room 32 A/B
ArcGIS Data Reviewer: Implementing Data Quality Reporting in Web Clients Demo Theater 10:30 – 11:15am Demo Theater 6
Server
ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 12:30 – 1:15pm Demo Theater 6
Server
Want to learn more this week?
ArcGIS Data Reviewer: An Introduction
Want to learn more?
• Product informationwww.esri.com/datareviewer
ArcGIS Data Reviewer: An Introduction
Want to learn more?
• Documentation- Desktop (desktop.arcgis.com)
ArcGIS Data Reviewer: An Introduction
Want to learn more?
• Documentation- Desktop- Server (server.arcgis.com)
ArcGIS Data Reviewer: An Introduction
Want to learn more?
• Documentation- Desktop- Server
• Training (training.esri.com)- Assessing Data Quality using ArcGIS Data Reviewer (Seminar)- Evaluating Positional Accuracy Using ArcGIS Data Reviewer for Desktop (Seminar)- Data QC with ArcGIS: Automating Validation (Web Course)- Data QC with ArcGIS: Visual Review (Web Course)- Quality Control Using ArcGIS Data Reviewer for Desktop (Instructor-Led)
ArcGIS Data Reviewer: An Introduction
Want to learn more?
• Documentation- Desktop- Server
• Training- Assessing Data Quality using ArcGIS Data Reviewer- Evaluating Positional Accuracy Using ArcGIS Data Reviewer for Desktop- Data QC with ArcGIS: Automating Validation- Data QC with ArcGIS: Visual Review- Quality Control Using ArcGIS Data Reviewer for Desktop
• GeoNet (geonet.esri.com)- Data Reviewer place
ArcGIS Data Reviewer: An Introduction