From Data to Display
From data to display Data loading, quality control and workflow management Kiyoshi Yamashita Kristen Waldvogel
Esri Software Components Production Mapping Data Review
Feature Manager
Workflow Management
Data Loader
Task Assistant Manager
Product Library
Workflow Manager Workflow Management
Task Assistant Manager
Data Reviewer Data Review
Data Interoperability Data Interoperability
Workflow Manager •Easily •Improve •Integrate
Manage Work
User Productivity
with GIS and Other Applications
Manage workflows at multiple levels
•
Macro – define business processes at a high level
•
Micro – tasks within an ArcMap session Workflow Manager Task Assistant
Anatomy of a Job Descriptive Information
Resource
Activity Log
Workflow Geographical Area of Interest
Geospatial data
Job Workflow •
Ensure that work is done the right way!
Start other processes
Concurrent Steps
Looping Workflows Decision Points
Email Stakeholders
Task Assistant Manager •
Standardize ArcMap workflows -
Reduce training time
-
Repeatable process
-
Simplify interface
•
Combine multiple tasks into one click
•
History
Workflow Manager Server • WMX web viewer – integrate with new and existing server solutions • Sample viewers available in Flex and JavaScript APIs
Data Interoperability •Data •Data •Data
Extraction
Translation
Loading of Standard GIS Formats
An Introduction to Data Interoperability
•
FME (Safe Software)
•
ETL Tool -
Extract: data from source
-
Transform: into desired format and schema
-
-
Create geometry
-
Concatenate/parse attributes
-
Remap domain values
-
Build relationships
Load: data into the destination system
Interacting with Data Interoperability Tools •
•
GP tool -
Run tool
-
Incorporate into Model Builder models, or call from GP Service
Workbench Edit Mode -
Build and edit tool
-
Run tool
ArcGIS Data Reviewer •Understanding •Automated •An
Data Reviewer
Review and Validation
Introduction to Data Reviewer Components
Managing Quality Control Scalable framework for managing QC lifecycle
A
REVIEW
CORRECT
VERIFY
Find & Record Errors
Perform Edits or Note Exceptions
Acceptable or Unacceptable
What is ArcGIS Data Reviewer? Data quality management for ArcGIS •
•
•
Provides -
Rule based workflows
-
Interactive tools
-
Track errors
Review
For individuals and enterprise -
Saves time/money
-
Less rework
Enterprise Workflow
Standard extension to ArcGIS -
Desktop extension since 8.0
-
Server extension at 10.1
Verify
Correct
Trusted data through improved quality management
An Introduction to Data Reviewer Components The Quality Control Process with Data Reviewer
Heart of thequality Reviewer • Stores spatial &Data attribute ••Summarize control • Automated Checks • information Similar to anabout edit session errors results • Tracks all anomalies/errors • Batch Data Review fromdata review • Personal, Allows interaction for File, orerror Enterprise • Interpret data QC• Visual purposes Geodatabase Review • Interactive • Report datatable accuracy • Feature class & tables created automatically
Data Checks The Reviewer Table Reviewer Session Report Quality Reviewer Workspace Information
A
New Capabilities for 10.1 Server-based Data Validation •
Automated quality control for the enterprise -
Scheduled and on-demand data validation
-
-
Scalable data validation capability based on ArcGIS Server
Frees data editors from running checks
ArcGIS Server
Business Rules
Batch Validation
Data Validation GP Service Error Results • Spatial extent • Properties
Reviewer Results Workspace
Production Workspace
Types of Quality Control
Automated Quality Control
Visual Review
Fast Consistent and repeatable Objective
100% coverage
As subjective as needed
Better for finding patterns and missing elements
Understanding Data Reviewer Checks
•
Over 40 data checks
•
Grouped into 11 categories
•
Configurable
•
Additive/Scriptable
www.esri.com/datareviewer
Combine Multiple Rules for Batch Check •
Encapsulate workflow / process
•
Designed once and executed many times
•
Complete spec check
An Introduction to Visual Data Review
•
•
Visual inspection of the data -
Level of judgment/discretion
-
Compare between data sources
-
Requires strong knowledge of the data & ability to make consistent decisions
Simplify visual review with Data Reviewer -
Organize and structure the visual review process
-
Manage error information in the Data Reviewer table
Quality Feedback via the Web Expanding the dialog on quality data management •
Extending quality control workflows into other communities -
QC review across ArcGIS platform
-
Simple to use tools for error identification
-
Deployed via intranet or Internet
Demo scenarios
Airport Local Government
Workflow Manager
Data Interoperability
Data Reviewer QC
√
Edit mode
Automated / Server
Geoprocessing tool
Server / Visual
Demo
Upcoming presentations
•
ArcGIS for Water Utilities -
2:45pm – 4:00pm - Data Reviewer for Infrastructure template for utilities applications
For more information…
•
resources.arcgis.com (Coming Soon) -
Mapping and Charting Data Quality
•
www.esri.com\WorkflowManager
•
www.esri.com\DataReviewer -
•
www.esri.com\DataInteroperability -
•
Product Information Demos Reviewer Poster
Transformers Supported Formats
Free 60-day software evaluations available
Questions?
Data Review
Workflow Management
Task Assistant Manager
Data Interoperability
Esri Software Components Production Mapping Data Review
Feature Manager
Workflow Management
Data Loader
Task Assistant Manager
Product Library
Workflow Manager Workflow Management
Task Assistant Manager
Data Reviewer Data Review
Data Interoperability Data Interoperability
Workflow Manager •Easily •Improve •Integrate
Manage Work
User Productivity
with GIS and Other Applications
Manage workflows at multiple levels
•
Macro – define business processes at a high level
•
Micro – tasks within an ArcMap session Workflow Manager Task Assistant
Anatomy of a Job Descriptive Information
Resource
Activity Log
Workflow Geographical Area of Interest
Geospatial data
Job Workflow •
Ensure that work is done the right way!
Start other processes
Concurrent Steps
Looping Workflows Decision Points
Email Stakeholders
Task Assistant Manager •
Standardize ArcMap workflows -
Reduce training time
-
Repeatable process
-
Simplify interface
•
Combine multiple tasks into one click
•
History
Workflow Manager Server • WMX web viewer – integrate with new and existing server solutions • Sample viewers available in Flex and JavaScript APIs
Data Interoperability •Data •Data •Data
Extraction
Translation
Loading of Standard GIS Formats
An Introduction to Data Interoperability
•
FME (Safe Software)
•
ETL Tool -
Extract: data from source
-
Transform: into desired format and schema
-
-
Create geometry
-
Concatenate/parse attributes
-
Remap domain values
-
Build relationships
Load: data into the destination system
Interacting with Data Interoperability Tools •
•
GP tool -
Run tool
-
Incorporate into Model Builder models, or call from GP Service
Workbench Edit Mode -
Build and edit tool
-
Run tool
ArcGIS Data Reviewer •Understanding •Automated •An
Data Reviewer
Review and Validation
Introduction to Data Reviewer Components
Managing Quality Control Scalable framework for managing QC lifecycle
A
REVIEW
CORRECT
VERIFY
Find & Record Errors
Perform Edits or Note Exceptions
Acceptable or Unacceptable
What is ArcGIS Data Reviewer? Data quality management for ArcGIS •
•
•
Provides -
Rule based workflows
-
Interactive tools
-
Track errors
Review
For individuals and enterprise -
Saves time/money
-
Less rework
Enterprise Workflow
Standard extension to ArcGIS -
Desktop extension since 8.0
-
Server extension at 10.1
Verify
Correct
Trusted data through improved quality management
An Introduction to Data Reviewer Components The Quality Control Process with Data Reviewer
Heart of thequality Reviewer • Stores spatial &Data attribute ••Summarize control • Automated Checks • information Similar to anabout edit session errors results • Tracks all anomalies/errors • Batch Data Review fromdata review • Personal, Allows interaction for File, orerror Enterprise • Interpret data QC• Visual purposes Geodatabase Review • Interactive • Report datatable accuracy • Feature class & tables created automatically
Data Checks The Reviewer Table Reviewer Session Report Quality Reviewer Workspace Information
A
New Capabilities for 10.1 Server-based Data Validation •
Automated quality control for the enterprise -
Scheduled and on-demand data validation
-
-
Scalable data validation capability based on ArcGIS Server
Frees data editors from running checks
ArcGIS Server
Business Rules
Batch Validation
Data Validation GP Service Error Results • Spatial extent • Properties
Reviewer Results Workspace
Production Workspace
Types of Quality Control
Automated Quality Control
Visual Review
Fast Consistent and repeatable Objective
100% coverage
As subjective as needed
Better for finding patterns and missing elements
Understanding Data Reviewer Checks
•
Over 40 data checks
•
Grouped into 11 categories
•
Configurable
•
Additive/Scriptable
www.esri.com/datareviewer
Combine Multiple Rules for Batch Check •
Encapsulate workflow / process
•
Designed once and executed many times
•
Complete spec check
An Introduction to Visual Data Review
•
•
Visual inspection of the data -
Level of judgment/discretion
-
Compare between data sources
-
Requires strong knowledge of the data & ability to make consistent decisions
Simplify visual review with Data Reviewer -
Organize and structure the visual review process
-
Manage error information in the Data Reviewer table
Quality Feedback via the Web Expanding the dialog on quality data management •
Extending quality control workflows into other communities -
QC review across ArcGIS platform
-
Simple to use tools for error identification
-
Deployed via intranet or Internet
Demo scenarios
Airport Local Government
Workflow Manager
Data Interoperability
Data Reviewer QC
√
Edit mode
Automated / Server
Geoprocessing tool
Server / Visual
Demo
Upcoming presentations
•
ArcGIS for Water Utilities -
2:45pm – 4:00pm - Data Reviewer for Infrastructure template for utilities applications
For more information…
•
resources.arcgis.com (Coming Soon) -
Mapping and Charting Data Quality
•
www.esri.com\WorkflowManager
•
www.esri.com\DataReviewer -
•
www.esri.com\DataInteroperability -
•
Product Information Demos Reviewer Poster
Transformers Supported Formats
Free 60-day software evaluations available
Questions?
Data Review
Workflow Management
Task Assistant Manager
Data Interoperability
