Enabling Effective Workflow and Data Quality Management
Tope Bello, Jay Cary, Javier Gutierrez April 11, 2012
Workshop Agenda
• Overview of ArcGIS Workflow Manager
• Overview of ArcGIS Data Reviewer
• Demonstration: Enabling Data Quality-focused Workflows
• Resources
• Questions
Overview of ArcGIS Workflow Manager
Section Topics
• Business Problem • ArcGIS Workflow Manager overview • System architecture • Extending Workflow Manager
Common Business Problems
• Why is managing work important?
- Optimize data production time - Improve data quality and accuracy - Reduce production cost - Resource allocation - Simplify processes
What is ArcGIS Workflow Manager Right work completed the right way, at the right time, by the right person
Standardize
Utilize Improve
Allows users to: • Standardize workflows
quickly
• Utilize workflows efficiently
• Improve performance continuously
Key Features and Capabilities
Anatomy of a Job
Geospatial data
Descriptive Information
Resource
Geographical Area of Interest
Activity Log
Workflow
Job Workflow
• Ensure that work is done the right way!
Concurrent Steps
Looping Workflows
Start other processes
Email Stakeholders
Decision Points
Real-world Workflow
Notifications
• Email notifications on - Job action - Workflow step - GIS data edits
• Fully configurable
Manager
GIS Analysts
Data Collection
Data Processing
Edit Data and Map
Perform Quality Check
Create the Final Output
Provide Project Status
• How long does the average project take to complete?
• What jobs are overdue?
• What areas needs more resources dedicated to them?
Tracking and Reporting Enforces Accountability
• What happened on this job? - Who did what? - When did it happen?
• Central information repository
Process Improvement
• Automate bottlenecks • Reallocate resources • Improve QA process
System Architecture
Workflow Manager System Framework
…Windows only
Workflow Manager
Geodatabase
Desktop
Administrator
Application
…Standard edition or higher
Server Web applications
SOAP/REST API
Why ArcGIS Workflow Manager Server?
Non-GIS Users • Customers • Admin staff
GIS Users • GIS analyst • QA • Data stewards
Applications • Maximo, SAP • In-house apps • SOA
GIS Department
Business Systems
Stake-holders
ArcGIS Workflow Manager Server
• Types of Services - Feature service - Geocode service - Geoprocessing service - Workflow Manager service - Etc.
ArcGIS Workflow Manager Services
SOAP - WSDL is available at
- http://<WebServer>:6080/<ArcGIS Instance>/services/<ServiceName>/WMServer?wsdl
REST - ArcGIS Server : http://<WebServer>:6080/<ArcGIS
Instance>/rest/services/<ServiceName>/WMServer
ArcGIS Workflow Manager Server Architecture
ArcGIS Server
Workflow Manager Server
Workflow Manager Server Applications
SOAP REST
Extending Workflow Manager
How can I extend Workflow Manager?
Desktop Server
• Commands • Toolbars • Tabs • Views • Applications
Workflows and External system integration
• Web Apps
Supported Development Platforms
• Custom Steps - .NET (C##, VB .NET) - C++ - Java
• Web Application - Flex, JavaScript, Silverlight, ASP .NET
Overview of ArcGIS Data Reviewer
Section Topics
• Importance of Data Quality
• What is ArcGIS Data Reviewer
• ArcGIS Data Reviewer for Server
• Enabling Data Quality-focused Workflows
Data Quality Matters
• Do you publish a doc without spell checking? • Do you operate a PC without a virus scanner?
• Cost of poor quality can be extreme
- $$ to fix - Impacts users - Reputation
What is ArcGIS Data Reviewer? Data quality management for ArcGIS
• Provides - Rule based workflows - Interactive tools - Track errors
• For individuals and enterprise - Saves time/money - Less rework
• Standard extension to ArcGIS - Desktop extension since 8.0 - Server extension since 10.1
Trusted data through improved quality management
Enterprise Workflow
Review
Correct Verify
REVIEW
Find & Record Errors
CORRECT
Perform Edits or
Note Exceptions
VERIFY (optional)
Acceptable or
Unacceptable
Managing Quality Control
• Framework for managing QC lifecycle
A
Support for the Complete System Make quality automatic and transparent
• Flexible automation of quality control for the organization
• Enable quality review and feedback from web clients
• Provide open access to data quality information
Cloud
Enterprise
Local
ArcGIS Data Reviewer Mobile
Desktop
Web
ArcGIS Data Reviewer Overview QC Workflows at 10.1
A
QC Workflows at 10
Visual QC
Automated Validation
DQ Reporting
Data Workspace(s)
Reviewer Workspace
Dashboard Reporting
Visual QC
Automated Validation
DQ Reporting Connected
Users
Server-based Adhoc/Scheduled
Data Workspace(s)
Reviewer Workspace
ArcGIS Data Reviewer for Server Capabilities
Rule-based Automated Data Review
• Over 40 data checks • Grouped into 11 categories • Configurable • Additive/Scriptable
www.esri.com/datareviewer
Batch Validation via the Web Leveraging servers for automated validation
• Automated quality control for the organization
- Scheduled and on-demand data validation
- Frees data editors from running checks
- Utilize servers for faster validation processing
Quality Feedback via the Web Expanding the dialog on quality data management
• Extending quality control workflows into other communities
- QC review across ArcGIS platform
- Simple to use tools for error identification
- Deployed via intranet or Internet
Manual quality control Expanding the dialog on data quality
Data Maintainers
A
Internal Feedback
• Consumers • Stakeholders • Volunteers
External Feedback
Internal Feedback
Dashboard Reporting Enabling transparency in data quality
• Better decision making by communicating data quality across stakeholders - Open quality reporting
- Shared across ArcGIS
system
- New tools and methods to communicate quality
DRS System Architecture
• Server Components - Server Object Extensions (SOE) - Geoprocessing (GP) Server - Accessible through REST API
• Web Components - Flex API - JavaScript API - Flex/JS Web applications
A
Dashboard
Flex Web App Flex Web App Flex/JS
Web Apps
Batch Validation
Custom Applications
REST API
Custom Flex/JS Applications
Flex/JS API
Workspace Browser
Enabling Data Quality-focused Workflows
Linking workflows with validation ArcGIS Data Reviewer for Desktop Integration
• Create Reviewer Session - Reviewer Workspace
• Execute Batch Job
- Reviewer Batch Job - Validation Area - Data Workspace - Reviewer Workspace - Reviewer Session Name
• Launch ArcMap - Reviewer Session
Linking workflows with validation ArcGIS Data Reviewer for Server Integration
• Server-based workflow integration emerging
• Future integration options: - Server-compatible WMX custom steps - Geoprocessing models/scripts - WMX Execute URL step
Demonstration: Enabling Data Quality-focused
Workflows
Scenario
• As a Contracting Officer I need to assess the quality of city-provided data before I can provide it to outside consultants for construction estimates.
• Acceptance Criteria: - Enable non-GIS user to identify areas where data is
required to support contractor bids - Automate independent review of data quality - Report errors detected in area of interest
Resources & Training
• Resource Center (resourcesbeta.arcgis.com) - Help and Documentation - Gallery (samples, tutorials, templates) - Forums - Blog
• Training (training.esri.com)
- Instructor Led Training - Web Courses - Training Seminars
Questions?
Please fill out the surveys on your way out.