Implementing ISO Data Quality Standards Using ESRI's GIS Data ...
Know the Earth…Show the Way
Implementing ISO Data Quality Standards Using ESRI’s GIS Data ReViewer
Dave Wesloh/John Sturley August 2004 ESRI User Conference NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Agenda • NGA/ESRI Cooperative Research and Development Agreement – What’s a CRADA – History of this CRADA • Goals and Objectives
• Geospatial Data – Data Overload – ISO Data Quality – GIS Data ReViewer 2 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • CRADA (Cooperative Research and Development Agreement) • CRADA Agreement signed by NGA (formerly NIMA) and ESRI April 1999 – Original Objectives • Jointly research training methods for data production using ESRI’s commercial-off-the-shelf (COTS) software and tools • Jointly research and enhance production processes that create NGA standard digital geospatial data and paper maps using ESRI's COTS software. 3 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • Research and enhance production processes. • GIS Data ReViewer software (COTS) – QC Tool enhancement (ArcView, PLTS) • Sampling, Edge Match, Point on Poly, etc. – Arc Info format – Shape file format – VPF format (VMap, DNC®, FFD)
4 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • ESRI and NGA determined need to extend CRADA based on several factors. – ESRI shifts from ArcInfo to ArcGIS – NGA shifts from VPF production to Country Databases – NGA shifts from QC to QA • QC - NGA checks the data • QA - The data producer (contractor) has implemented a quality process which when followed results in a quality product. NGA verifies the process not the products.
5 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • 3 Year Extension signed April 2002 • New CRADA Task: “Quality as defined within the ISO Standards is a factor of data completeness, logical consistency, positional accuracy, temporal accuracy and thematic accuracy. NGA currently has few systems or procedures for determining quality based on these factors. ESRI and NGA will research the requirements and functionality needed to address each of these Data Quality elements.” 6 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Data Overload • How to tell good data from bad data? • Need to understand the Quality of the data – What is it? – How do you measure it? – How can you understand the value provided by a quality measurement?
• Does the data meet your intended use?
7 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Data Quality • ISO 19113 – Quality Principles – Geographic datasets are increasingly being shared, interchanged and used for purposes other than their producers’ intended ones. Information about the quality of available geographic datasets is vital to the process of selecting a dataset in that the value of data is directly related to its quality.
• ISO 19114 – Quality Evaluation Procedures – For the purpose of evaluating the quality of a dataset, clearly defined procedures must be used in a consistent manner. This enables data producers to express how well their product meets the criteria set forth in its product specification and data users to establish the extent to which a dataset meets their requirements.
8 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Data Quality • ISO 19113 Quality Principles – Positional Accuracy – Logical Consistency – Thematic Accuracy – Completeness – Temporal Accuracy
Attributes
Features
Themes
Dataset
Scope versus Universe of Discourse (UOD) 9 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Geospatial Data Components for Quality Evaluation • Dataset – Structure – Spatial • Geometry • Geometric relationships (topology)
– Features • Attributes • Attribute values
– Thematic layers (feature groupings)
10 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Positional Accuracy • Absolute – Horizontal and Vertical – closeness of reported coordinate values to values accepted as being true
• Relative – Point to Point – closeness of the relative positions of features in a dataset to their respective relative positions accepted as being true 11 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer Building corner
Road Intersection
Positional Accuracy Absolute Circular Error Data to Image Comparison Dam edge
Road/River Intersection MODIS IMAGE Image courtesy MODIS (Moderate Resolution Imaging Spectroradiometer) Web site
12 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer Positional Accuracy
Relative Accuracy Feature to Feature
MODIS IMAGE Image courtesy MODIS (Moderate Resolution Imaging Spectroradiometer) Web site
13 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Positional Accuracy Assessment Tool – Works in conjunction with BAE Socet Set or ERDAS Imagine – Calculates at 90% and 95% confidence level – Compares raster to raster, raster to vector, vector to vector – Uses stereo images, ortho-rectified mono images or geodetic control points as source – Populates dataset level metadata and/or feature level positional accuracy attributes
14 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Logical Consistency • Logical Consistency – – – –
Conceptual Consistency Domain Consistency Format Consistency Topological Consistency
15 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Conceptual Consistency: – Verify data content matches Conceptual Schema desired. – The Conceptual Schema can be a product specification or a set of User Defined Requirements. – This will be a Pass/Fail test – For example if the Universe of Discourse (UOD) is defined by a VPF specification, and the Scope (dataset) is an Arc Shapefile, this test would result in a Fail. 16 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer
• Format Consistency - Feature Coding and structure requirements compare to requested format Tunnels - AQ130 (DGIWG FACC Coding) Feature Code: Length: Name: TUC:
Text Field (5 Characters) Short Integer Text Field (variable length) Short Integer
• Reporting will be done on a Percent of Violating Items
17
Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Domain Consistency - Attribute values that fall outside the domain as established by the UOD Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
• Reporting will be done on a Percent of Violating Items 18 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Topological Consistency – Implements three types of topological evaluations, captures information from ArcGIS and GIS Data ReViewer, report on Percent of Violating Items: • Conformance to standard definitions and conventions for point (i.e. single vertex), line (i.e. minimum of 2 vertices, contiguous), and area features (i.e. minimum of 4 vertices, contiguous) – ArcGIS, Validate Simple Geometry (duplicate vertices, minimum area • Relational conformance (i.e. nodes at intersections, overshoots, undershoots, slivers, etc.) – ArcGIS, Duplicate Point, Sliver Check, Validate Simple Geometry (find overlaps, crosses-self), dangle renderer • Feature to Feature conformance (i.e. bridge to road connectivity, buildings inside lakes, etc) – ArcGIS, Point On Poly, , Validate Simple Geometry (crosses-others), Batch Validate (condition tables) 19 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Thematic Accuracy • Classification Correctness • Non Quantitative Attribute Correctness • Quantitative Attribute Correctness
20 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Classification Correctness - Compare data against UOD to verify the correct classification of features and their relationships Thematic Layer: Transportation Feature Code: Attributes: Feature Code: …….. Feature Code: …….. Feature Code: ……..
AQ130 (tunnel) LEN, NAM, TUC AP010 (cart track) AP030 (road) AN010 (railroad)
• Reporting will be done on a Percent of Violating Items 21 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Non Quantitative Attribute Correctness – Reporting on % of Non quantitative attributes found in error – Based on sampling or Compare tool if specification driven Thematic Layer: Transportation Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
22 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Quantitative Attribute Correctness – Report on % of quantitative attributes found in error – Based on sampling Thematic Layer: Transportation Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
23 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Completeness • Completeness: presence and absence of features, their attributes and relationships; Completeness COMMISSION: – Feature present that doesn’t belong to the UOD – Attribute present that doesn’t belong to the UOD
Completeness OMISSION: – Feature in UOD not found in Scope (not necessarily an error) – Feature in UOD and Scope but Attribute doesn’t match
• Sum by F_CODE, Frequency, Data Loader • Report will be on Percentage of extra or missing items 24 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Tool will run in an automated method when UOD is a specification • A user defined UOD is under consideration GIFD
User defined x-ref
Feature Database x-ref
f e r x-
MSD LV1 MSD LV2
xre
f
MSD LV3 25 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Visual evaluation of data density
= heavy = medium = light
26 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Reporting Quality • Provide a “Data Quality Report”, automatically populate metadata fields, and create a data quality coverage . • Error Shapefile (points) • Error Table
27 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Summary • Substantial amount of work still to be done. • A lot has been accomplished. – Positional Accuracy Assessment Tool – Some work on remaining DQ elements
• NGA requires Data Quality reporting. capabilities. • NGA will use ISO for DQ guidelines. • GIS Data ReViewer is our best option for success. 28 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Know the Earth…Show the Way
29 Know the Earth…Show the Way
Implementing ISO Data Quality Standards Using ESRI’s GIS Data ReViewer
Dave Wesloh/John Sturley August 2004 ESRI User Conference NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Agenda • NGA/ESRI Cooperative Research and Development Agreement – What’s a CRADA – History of this CRADA • Goals and Objectives
• Geospatial Data – Data Overload – ISO Data Quality – GIS Data ReViewer 2 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • CRADA (Cooperative Research and Development Agreement) • CRADA Agreement signed by NGA (formerly NIMA) and ESRI April 1999 – Original Objectives • Jointly research training methods for data production using ESRI’s commercial-off-the-shelf (COTS) software and tools • Jointly research and enhance production processes that create NGA standard digital geospatial data and paper maps using ESRI's COTS software. 3 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • Research and enhance production processes. • GIS Data ReViewer software (COTS) – QC Tool enhancement (ArcView, PLTS) • Sampling, Edge Match, Point on Poly, etc. – Arc Info format – Shape file format – VPF format (VMap, DNC®, FFD)
4 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • ESRI and NGA determined need to extend CRADA based on several factors. – ESRI shifts from ArcInfo to ArcGIS – NGA shifts from VPF production to Country Databases – NGA shifts from QC to QA • QC - NGA checks the data • QA - The data producer (contractor) has implemented a quality process which when followed results in a quality product. NGA verifies the process not the products.
5 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NGA/ESRI CRADA • 3 Year Extension signed April 2002 • New CRADA Task: “Quality as defined within the ISO Standards is a factor of data completeness, logical consistency, positional accuracy, temporal accuracy and thematic accuracy. NGA currently has few systems or procedures for determining quality based on these factors. ESRI and NGA will research the requirements and functionality needed to address each of these Data Quality elements.” 6 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Data Overload • How to tell good data from bad data? • Need to understand the Quality of the data – What is it? – How do you measure it? – How can you understand the value provided by a quality measurement?
• Does the data meet your intended use?
7 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Data Quality • ISO 19113 – Quality Principles – Geographic datasets are increasingly being shared, interchanged and used for purposes other than their producers’ intended ones. Information about the quality of available geographic datasets is vital to the process of selecting a dataset in that the value of data is directly related to its quality.
• ISO 19114 – Quality Evaluation Procedures – For the purpose of evaluating the quality of a dataset, clearly defined procedures must be used in a consistent manner. This enables data producers to express how well their product meets the criteria set forth in its product specification and data users to establish the extent to which a dataset meets their requirements.
8 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Data Quality • ISO 19113 Quality Principles – Positional Accuracy – Logical Consistency – Thematic Accuracy – Completeness – Temporal Accuracy
Attributes
Features
Themes
Dataset
Scope versus Universe of Discourse (UOD) 9 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Geospatial Data Components for Quality Evaluation • Dataset – Structure – Spatial • Geometry • Geometric relationships (topology)
– Features • Attributes • Attribute values
– Thematic layers (feature groupings)
10 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Positional Accuracy • Absolute – Horizontal and Vertical – closeness of reported coordinate values to values accepted as being true
• Relative – Point to Point – closeness of the relative positions of features in a dataset to their respective relative positions accepted as being true 11 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer Building corner
Road Intersection
Positional Accuracy Absolute Circular Error Data to Image Comparison Dam edge
Road/River Intersection MODIS IMAGE Image courtesy MODIS (Moderate Resolution Imaging Spectroradiometer) Web site
12 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer Positional Accuracy
Relative Accuracy Feature to Feature
MODIS IMAGE Image courtesy MODIS (Moderate Resolution Imaging Spectroradiometer) Web site
13 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Positional Accuracy Assessment Tool – Works in conjunction with BAE Socet Set or ERDAS Imagine – Calculates at 90% and 95% confidence level – Compares raster to raster, raster to vector, vector to vector – Uses stereo images, ortho-rectified mono images or geodetic control points as source – Populates dataset level metadata and/or feature level positional accuracy attributes
14 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Logical Consistency • Logical Consistency – – – –
Conceptual Consistency Domain Consistency Format Consistency Topological Consistency
15 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Conceptual Consistency: – Verify data content matches Conceptual Schema desired. – The Conceptual Schema can be a product specification or a set of User Defined Requirements. – This will be a Pass/Fail test – For example if the Universe of Discourse (UOD) is defined by a VPF specification, and the Scope (dataset) is an Arc Shapefile, this test would result in a Fail. 16 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer
• Format Consistency - Feature Coding and structure requirements compare to requested format Tunnels - AQ130 (DGIWG FACC Coding) Feature Code: Length: Name: TUC:
Text Field (5 Characters) Short Integer Text Field (variable length) Short Integer
• Reporting will be done on a Percent of Violating Items
17
Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Domain Consistency - Attribute values that fall outside the domain as established by the UOD Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
• Reporting will be done on a Percent of Violating Items 18 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Topological Consistency – Implements three types of topological evaluations, captures information from ArcGIS and GIS Data ReViewer, report on Percent of Violating Items: • Conformance to standard definitions and conventions for point (i.e. single vertex), line (i.e. minimum of 2 vertices, contiguous), and area features (i.e. minimum of 4 vertices, contiguous) – ArcGIS, Validate Simple Geometry (duplicate vertices, minimum area • Relational conformance (i.e. nodes at intersections, overshoots, undershoots, slivers, etc.) – ArcGIS, Duplicate Point, Sliver Check, Validate Simple Geometry (find overlaps, crosses-self), dangle renderer • Feature to Feature conformance (i.e. bridge to road connectivity, buildings inside lakes, etc) – ArcGIS, Point On Poly, , Validate Simple Geometry (crosses-others), Batch Validate (condition tables) 19 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Thematic Accuracy • Classification Correctness • Non Quantitative Attribute Correctness • Quantitative Attribute Correctness
20 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Classification Correctness - Compare data against UOD to verify the correct classification of features and their relationships Thematic Layer: Transportation Feature Code: Attributes: Feature Code: …….. Feature Code: …….. Feature Code: ……..
AQ130 (tunnel) LEN, NAM, TUC AP010 (cart track) AP030 (road) AN010 (railroad)
• Reporting will be done on a Percent of Violating Items 21 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Non Quantitative Attribute Correctness – Reporting on % of Non quantitative attributes found in error – Based on sampling or Compare tool if specification driven Thematic Layer: Transportation Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
22 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Quantitative Attribute Correctness – Report on % of quantitative attributes found in error – Based on sampling Thematic Layer: Transportation Feature Code: AQ130 Length: 0 (unknown) >= 75 meters Name: Character String “UNK” no name present TUC: 0 (unknown) 1 Both Road and Railroad 3 Railroad 4 Road 38 Canal
23 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
ISO Completeness • Completeness: presence and absence of features, their attributes and relationships; Completeness COMMISSION: – Feature present that doesn’t belong to the UOD – Attribute present that doesn’t belong to the UOD
Completeness OMISSION: – Feature in UOD not found in Scope (not necessarily an error) – Feature in UOD and Scope but Attribute doesn’t match
• Sum by F_CODE, Frequency, Data Loader • Report will be on Percentage of extra or missing items 24 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Tool will run in an automated method when UOD is a specification • A user defined UOD is under consideration GIFD
User defined x-ref
Feature Database x-ref
f e r x-
MSD LV1 MSD LV2
xre
f
MSD LV3 25 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
GIS Data ReViewer • Visual evaluation of data density
= heavy = medium = light
26 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Reporting Quality • Provide a “Data Quality Report”, automatically populate metadata fields, and create a data quality coverage . • Error Shapefile (points) • Error Table
27 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Summary • Substantial amount of work still to be done. • A lot has been accomplished. – Positional Accuracy Assessment Tool – Some work on remaining DQ elements
• NGA requires Data Quality reporting. capabilities. • NGA will use ISO for DQ guidelines. • GIS Data ReViewer is our best option for success. 28 Know the Earth…Show the Way
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY
Know the Earth…Show the Way
29 Know the Earth…Show the Way
