enabling effective workflow and data quality management

Enabling Effective Workflow and Data Quality Management

Tope Bello, Jay Cary, Javier Gutierrez April 11, 2012

Workshop Agenda

• Overview of ArcGIS Workflow Manager

• Overview of ArcGIS Data Reviewer

• Demonstration: Enabling Data Quality-focused Workflows

• Resources

• Questions

Overview of ArcGIS Workflow Manager

Section Topics

• Business Problem • ArcGIS Workflow Manager overview • System architecture • Extending Workflow Manager

Common Business Problems

• Why is managing work important?

- Optimize data production time - Improve data quality and accuracy - Reduce production cost - Resource allocation - Simplify processes

What is ArcGIS Workflow Manager Right work completed the right way, at the right time, by the right person

Standardize

Utilize Improve

Allows users to: • Standardize workflows

quickly

• Utilize workflows efficiently

• Improve performance continuously

Key Features and Capabilities

Anatomy of a Job

Geospatial data

Descriptive Information

Resource

Geographical Area of Interest

Activity Log

Workflow

Job Workflow

• Ensure that work is done the right way!

Concurrent Steps

Looping Workflows

Start other processes

Email Stakeholders

Decision Points

Real-world Workflow

Notifications

• Email notifications on - Job action - Workflow step - GIS data edits

• Fully configurable

Manager

GIS Analysts

Data Collection

Data Processing

Edit Data and Map

Perform Quality Check

Create the Final Output

Provide Project Status

• How long does the average project take to complete?

• What jobs are overdue?

• What areas needs more resources dedicated to them?

Tracking and Reporting Enforces Accountability

• What happened on this job? - Who did what? - When did it happen?

• Central information repository

Process Improvement

• Automate bottlenecks • Reallocate resources • Improve QA process

System Architecture

Workflow Manager System Framework

…Windows only

Workflow Manager

Geodatabase

Desktop

Administrator

Application

…Standard edition or higher

Server Web applications

SOAP/REST API

Why ArcGIS Workflow Manager Server?

Non-GIS Users • Customers • Admin staff

GIS Users • GIS analyst • QA • Data stewards

Applications • Maximo, SAP • In-house apps • SOA

GIS Department

Business Systems

Stake-holders

ArcGIS Workflow Manager Server

• Types of Services - Feature service - Geocode service - Geoprocessing service - Workflow Manager service - Etc.

ArcGIS Workflow Manager Services

SOAP - WSDL is available at

- http://<WebServer>:6080/<ArcGIS Instance>/services/<ServiceName>/WMServer?wsdl

REST - ArcGIS Server : http://<WebServer>:6080/<ArcGIS

Instance>/rest/services/<ServiceName>/WMServer

ArcGIS Workflow Manager Server Architecture

ArcGIS Server

Workflow Manager Server

Workflow Manager Server Applications

SOAP REST

Extending Workflow Manager

How can I extend Workflow Manager?

Desktop Server

• Commands • Toolbars • Tabs • Views • Applications

Workflows and External system integration

• Web Apps

Supported Development Platforms

• Custom Steps - .NET (C##, VB .NET) - C++ - Java

• Web Application - Flex, JavaScript, Silverlight, ASP .NET

Overview of ArcGIS Data Reviewer

Section Topics

• Importance of Data Quality

• What is ArcGIS Data Reviewer

• ArcGIS Data Reviewer for Server

• Enabling Data Quality-focused Workflows

Data Quality Matters

• Do you publish a doc without spell checking? • Do you operate a PC without a virus scanner?

• Cost of poor quality can be extreme

- $$ to fix - Impacts users - Reputation

What is ArcGIS Data Reviewer? Data quality management for ArcGIS

• Provides - Rule based workflows - Interactive tools - Track errors

• For individuals and enterprise - Saves time/money - Less rework

• Standard extension to ArcGIS - Desktop extension since 8.0 - Server extension since 10.1

Trusted data through improved quality management

Enterprise Workflow

Review

Correct Verify

REVIEW

Find & Record Errors

CORRECT

Perform Edits or

Note Exceptions

VERIFY (optional)

Acceptable or

Unacceptable

Managing Quality Control

• Framework for managing QC lifecycle

A

Support for the Complete System Make quality automatic and transparent

• Flexible automation of quality control for the organization

• Enable quality review and feedback from web clients

• Provide open access to data quality information

Cloud

Enterprise

Local

ArcGIS Data Reviewer Mobile

Desktop

Web

ArcGIS Data Reviewer Overview QC Workflows at 10.1

A

QC Workflows at 10

Visual QC

Automated Validation

DQ Reporting

Data Workspace(s)

Reviewer Workspace

Dashboard Reporting

Visual QC

Automated Validation

DQ Reporting Connected

Users

Server-based Adhoc/Scheduled

Data Workspace(s)

Reviewer Workspace

ArcGIS Data Reviewer for Server Capabilities

Rule-based Automated Data Review

• Over 40 data checks • Grouped into 11 categories • Configurable • Additive/Scriptable

www.esri.com/datareviewer

Batch Validation via the Web Leveraging servers for automated validation

• Automated quality control for the organization

- Scheduled and on-demand data validation

- Frees data editors from running checks

- Utilize servers for faster validation processing

Quality Feedback via the Web Expanding the dialog on quality data management

• Extending quality control workflows into other communities

- QC review across ArcGIS platform

- Simple to use tools for error identification

- Deployed via intranet or Internet

Manual quality control Expanding the dialog on data quality

Data Maintainers

A

Internal Feedback

• Consumers • Stakeholders • Volunteers

External Feedback

Internal Feedback

Dashboard Reporting Enabling transparency in data quality

• Better decision making by communicating data quality across stakeholders - Open quality reporting

- Shared across ArcGIS

system

- New tools and methods to communicate quality

DRS System Architecture

• Server Components - Server Object Extensions (SOE) - Geoprocessing (GP) Server - Accessible through REST API

• Web Components - Flex API - JavaScript API - Flex/JS Web applications

A

Dashboard

Flex Web App Flex Web App Flex/JS

Web Apps

Batch Validation

Custom Applications

REST API

Custom Flex/JS Applications

Flex/JS API

Workspace Browser

Enabling Data Quality-focused Workflows

Linking workflows with validation ArcGIS Data Reviewer for Desktop Integration

• Create Reviewer Session - Reviewer Workspace

• Execute Batch Job

- Reviewer Batch Job - Validation Area - Data Workspace - Reviewer Workspace - Reviewer Session Name

• Launch ArcMap - Reviewer Session

Linking workflows with validation ArcGIS Data Reviewer for Server Integration

• Server-based workflow integration emerging

• Future integration options: - Server-compatible WMX custom steps - Geoprocessing models/scripts - WMX Execute URL step

Demonstration: Enabling Data Quality-focused

Workflows

Scenario

• As a Contracting Officer I need to assess the quality of city-provided data before I can provide it to outside consultants for construction estimates.

• Acceptance Criteria: - Enable non-GIS user to identify areas where data is

required to support contractor bids - Automate independent review of data quality - Report errors detected in area of interest

Resources & Training

• Resource Center (resourcesbeta.arcgis.com) - Help and Documentation - Gallery (samples, tutorials, templates) - Forums - Blog

• Training (training.esri.com)

- Instructor Led Training - Web Courses - Training Seminars

Questions?

Please fill out the surveys on your way out.