Image Services for Elevation Data
Image Services for Elevation Data Peter Becker
Need for Elevation Using Image Services for Elevation Data sources Creating Elevation Service
Requirement: GIS and Imagery, Integrated and Accessible Decision Makers
Field Users
Analysts
Data Managers
GIS • Actionable information and knowledge • Unified geospatial workflows
Imagery
+
Sensor Networks
+
Knowledge Base
Base Maps & Globes
Georeferenced Content
Uses of Elevation Data
• • • • • • • • • •
As Elevation 3D Visualization Shaded Relief – cartographic Aspect – Agriculture Slope – Land subsidence, off-road mobility analysis Profiles – Planning pipelines, drainage Viewshed – Visibility analysis Orthorectification Difference/Volume – Tree Height Change – Ground movement
Traditional Elevation Management
• Analyst work on individual projects • Merge together different sources • Similar to how imagery was managed
Elevation Data Requirements
• Multiple: – Resolution – Global to Local – Source – SRTM to Lidar – Format – Many formats used – Projection – Multiple Horizontal and Vertical Datums
Sources
• • • • • • • • • •
GTOP – 30-arc second (1km) SRTM – Shuttle Mission Topography Mission - 3-arc second (90m) ASTER – GDEM – 1-arc second (30m) NED – 1-arc second (30m) & 1/3-arc second (10m) Intermap, SPOT, …. Photogrammery – Correlation or point capture Lidar Bathymetry Sonar From Contour, vector sources
Varying accuracies (VE90 CE90)
Demo – Using Image services for elevation
Formats • Irregular – LAS – ASCII XYZ – 3D SHP – CAD –…
• Gridded – TIFF – ESRI Grid – ASCII – DTED – IMG – JPEG2000 –…
LiDaR (Light Detection and Ranging)
• • • • •
Laser based High Density, High Accuracy Partially penetrates foliage LAS 1.2 - ASPRS Include classification
Terrain Dataset Multi-resolution surface created from measurements stored in feature classes
• MassPoints (Lidar), breaklines, spot heights, polygons, …
• • • • • • •
Stored in the geodatabase Schema: Defines feature class participation On-The-Fly TIN Multi-resolution Highly scalable Attributes Editable, Versioned Points and Breaklines Terrain Pyramids
High Resolution
Medium Resolution
Low Resolution
Raster / Grid For Elevation Generally a Derivative
• Advantages – Simplest representation – Highly scalable – Fast access
• Disadvantages – Loss of original point data – Require multiple rasters for different classifications, attributes
• Terrain can be exported as Raster – Use ‘Terrain to Raster’ – Consider exporting at multiple pixel sizes
No Data Areas • Areas with No Valid values • Where possible interpolate values • Consider Accuracy Implications • Clip using Footprints • Define as No Data Value • Define Mask Selection of Pixels by footprint intersect
Selecting of Pixels by Analyzing transparent pixels
Profile View
Lidar to Raster Conversion • Via Terrain – Point File Information (Average point spacing) – Terrain Wizard – With Terrain, holes are filled by TIN – Terrain to Raster
• Interpolate to Raster – LAS to Multi-point – Point to Raster • Set Point Set cell size to 4x avg. point spacing
– Filter to remove holes • Con(IsNULL([inputraster]), FOCALMEAN ([inputraster], RECTANGLE, 3, 3, data), [inputraster])
• 3rd Party Tools – Eg LP360 (QCoherent)
• Considerations – Have overlap in tiles (20 pixels) – Obtain extents of Lidar
Conversion of Raster • Conversion sometime required for optimization • Optimal format – TIF – LZW compressed, 16 or 32bit
• Internal overviews or .OVR • ArcGIS Copy Raster
• GDAL (FWTools.org) GDALTranslate -of Gtiff -co “COMPRESS=LZW” -co “PREDICTOR=2” input output.tif GDALAddo -r average --config COMPRESS_OVERVIEW LZW --config USE_RRD NO --config PREDICTOR 2 output.tif 2 4 8 16 32
Image Services Optimized Access to Imagery
• • • • •
New at ArcGIS Server 9.3 Serves all supported raster data (TIF, IMG, MrSID, …) Web accessible via SOAP, REST, OGC (WMS, WCS, KML) Accessible as image and pixel values Server side rendering enables, client definition of: – – –
Projection Compression for transmission Sampling methods
• Can be used to serve elevation data
ArcGIS Server - Image Extension Extends ArcGIS Server for Large Numbers of Images • Manage large collections of images • Provide single point of access • Dynamic mosaicking • On-the-fly processing • Maintain source information for every pixel • Simplified data management
Use
Serve
ArcGIS Server Image Extension
• Combines processing with serving • Keep imagery in native form • Author service definition (catalog) • Properties and processes • Publish as Image Service • Client controllable properties
Author
Service Service Definition Service Definition Definition
Dynamic Mosaicking Fuses images as they are accessed
Resolves Issues with Traditional Image Management
Additional Storage
Processing Time
Mosaicking requires storage
Time consuming
Multi-resolution Data
Overlapping Imagery Loss of information
t1 t2 t3 t4 t5
Need to sample up or down
Disparate Datasets Large NoData areas
Maintenance Difficulty making changes
Loss of Image Quality Sampling degrades quality
Loss of Metadata Valuable information
Very applicable for Elevation Data
On-The-Fly Processing Create Multiple Products from a Single Source • Processing can be applied to raster or service level • Processes • • • • • • • •
Enhance Extract Bands Orthorectify PanSharpen Vegetation Index Classify Shaded Relief Crop to Footprint
• Visualize Elevation • ...
Enable the creation of derived products with no additional storage or pre-processing
Using Image Extension with Elevation Data
• Create Single Image Service – Of all raster Elevation Sources – Multi-Resolution
• Create Derived Service that define multiple products – Slope – Aspect – Shaded Relief – Hill Shade
• Accessible to a wide range of applications • Usable in GeoProcessing
Considerations when Creating Services
• • • • • • •
Possibly Optimize Imagery Do NOT Reproject or sample data (will be done OTF) Recompute footprints to remove NoData Use Calculate to Set MinPS and MaxPS Use DispOrder or SysOrder to force order Consider location of processes – Use Edit Raster Properties Consider adding attributes to define accuracy
Demo
• Create Image Service Definition from – GTOP – SRTM – NED 30m – NED 10m – Rasterized Lidar Data
Creating Derived Services
• • • •
Enable Multiple Service from same source Enables rendering: EG for WMS Enables high compression for transmission Simplifies data management
• Use SaveAs, or reference using ISRefs • Developer can use client defined processes
Using Multiple Services • Height = (First Return – Bare Earth)
• Correct Geoid
GeoProcessing
• In ArcGIS Desktop – Export AOI at required resolution
• In Geoprocessing Tools – Use Make Image Server Layer
9.3.1 Limitations – With Image Services • Transparency of No Data not handled well – Use footprints
• • • •
Black lines at edges of export with focal processes Source Vertical Datum Different renderer then ArcGIS – Solution: Use as Layer on ArcGIS Server
ArcGIS 10 – Improvements for Imagery and Elevation
• Management – Mosaic Dataset – Stored in GeoDatabase
• Access – More formats
• Improved No Data Support • Image Analysis Window – Access OTF processing in Desktop
• Spatial Analysis – Direct use of multiple formats – Improved speed
ArcGIS The Platform for Fully Integrated GIS and Imagery • • • •
Integrating Imagery as core to GIS Management, Dissemination, Visualization and Analysis Solution for wide range of user requirements Maximizes the value of imagery including Imagery
• • •
Single Source for all elevation Multiple Products from a single source Maintainability
GIS
++
Learn More http://www.esri.com/training
• Instructor-Led Training – Introduction to ArcGIS Server Image Extension – ArcGIS Server: Web Administration Using the Microsoft .NET Framework
• Free Web Training Seminar – Serving and Managing Imagery with ArcGIS
ESRI Training…keep critical skills up to date
Need for Elevation Using Image Services for Elevation Data sources Creating Elevation Service
Requirement: GIS and Imagery, Integrated and Accessible Decision Makers
Field Users
Analysts
Data Managers
GIS • Actionable information and knowledge • Unified geospatial workflows
Imagery
+
Sensor Networks
+
Knowledge Base
Base Maps & Globes
Georeferenced Content
Uses of Elevation Data
• • • • • • • • • •
As Elevation 3D Visualization Shaded Relief – cartographic Aspect – Agriculture Slope – Land subsidence, off-road mobility analysis Profiles – Planning pipelines, drainage Viewshed – Visibility analysis Orthorectification Difference/Volume – Tree Height Change – Ground movement
Traditional Elevation Management
• Analyst work on individual projects • Merge together different sources • Similar to how imagery was managed
Elevation Data Requirements
• Multiple: – Resolution – Global to Local – Source – SRTM to Lidar – Format – Many formats used – Projection – Multiple Horizontal and Vertical Datums
Sources
• • • • • • • • • •
GTOP – 30-arc second (1km) SRTM – Shuttle Mission Topography Mission - 3-arc second (90m) ASTER – GDEM – 1-arc second (30m) NED – 1-arc second (30m) & 1/3-arc second (10m) Intermap, SPOT, …. Photogrammery – Correlation or point capture Lidar Bathymetry Sonar From Contour, vector sources
Varying accuracies (VE90 CE90)
Demo – Using Image services for elevation
Formats • Irregular – LAS – ASCII XYZ – 3D SHP – CAD –…
• Gridded – TIFF – ESRI Grid – ASCII – DTED – IMG – JPEG2000 –…
LiDaR (Light Detection and Ranging)
• • • • •
Laser based High Density, High Accuracy Partially penetrates foliage LAS 1.2 - ASPRS Include classification
Terrain Dataset Multi-resolution surface created from measurements stored in feature classes
• MassPoints (Lidar), breaklines, spot heights, polygons, …
• • • • • • •
Stored in the geodatabase Schema: Defines feature class participation On-The-Fly TIN Multi-resolution Highly scalable Attributes Editable, Versioned Points and Breaklines Terrain Pyramids
High Resolution
Medium Resolution
Low Resolution
Raster / Grid For Elevation Generally a Derivative
• Advantages – Simplest representation – Highly scalable – Fast access
• Disadvantages – Loss of original point data – Require multiple rasters for different classifications, attributes
• Terrain can be exported as Raster – Use ‘Terrain to Raster’ – Consider exporting at multiple pixel sizes
No Data Areas • Areas with No Valid values • Where possible interpolate values • Consider Accuracy Implications • Clip using Footprints • Define as No Data Value • Define Mask Selection of Pixels by footprint intersect
Selecting of Pixels by Analyzing transparent pixels
Profile View
Lidar to Raster Conversion • Via Terrain – Point File Information (Average point spacing) – Terrain Wizard – With Terrain, holes are filled by TIN – Terrain to Raster
• Interpolate to Raster – LAS to Multi-point – Point to Raster • Set Point Set cell size to 4x avg. point spacing
– Filter to remove holes • Con(IsNULL([inputraster]), FOCALMEAN ([inputraster], RECTANGLE, 3, 3, data), [inputraster])
• 3rd Party Tools – Eg LP360 (QCoherent)
• Considerations – Have overlap in tiles (20 pixels) – Obtain extents of Lidar
Conversion of Raster • Conversion sometime required for optimization • Optimal format – TIF – LZW compressed, 16 or 32bit
• Internal overviews or .OVR • ArcGIS Copy Raster
• GDAL (FWTools.org) GDALTranslate -of Gtiff -co “COMPRESS=LZW” -co “PREDICTOR=2” input output.tif GDALAddo -r average --config COMPRESS_OVERVIEW LZW --config USE_RRD NO --config PREDICTOR 2 output.tif 2 4 8 16 32
Image Services Optimized Access to Imagery
• • • • •
New at ArcGIS Server 9.3 Serves all supported raster data (TIF, IMG, MrSID, …) Web accessible via SOAP, REST, OGC (WMS, WCS, KML) Accessible as image and pixel values Server side rendering enables, client definition of: – – –
Projection Compression for transmission Sampling methods
• Can be used to serve elevation data
ArcGIS Server - Image Extension Extends ArcGIS Server for Large Numbers of Images • Manage large collections of images • Provide single point of access • Dynamic mosaicking • On-the-fly processing • Maintain source information for every pixel • Simplified data management
Use
Serve
ArcGIS Server Image Extension
• Combines processing with serving • Keep imagery in native form • Author service definition (catalog) • Properties and processes • Publish as Image Service • Client controllable properties
Author
Service Service Definition Service Definition Definition
Dynamic Mosaicking Fuses images as they are accessed
Resolves Issues with Traditional Image Management
Additional Storage
Processing Time
Mosaicking requires storage
Time consuming
Multi-resolution Data
Overlapping Imagery Loss of information
t1 t2 t3 t4 t5
Need to sample up or down
Disparate Datasets Large NoData areas
Maintenance Difficulty making changes
Loss of Image Quality Sampling degrades quality
Loss of Metadata Valuable information
Very applicable for Elevation Data
On-The-Fly Processing Create Multiple Products from a Single Source • Processing can be applied to raster or service level • Processes • • • • • • • •
Enhance Extract Bands Orthorectify PanSharpen Vegetation Index Classify Shaded Relief Crop to Footprint
• Visualize Elevation • ...
Enable the creation of derived products with no additional storage or pre-processing
Using Image Extension with Elevation Data
• Create Single Image Service – Of all raster Elevation Sources – Multi-Resolution
• Create Derived Service that define multiple products – Slope – Aspect – Shaded Relief – Hill Shade
• Accessible to a wide range of applications • Usable in GeoProcessing
Considerations when Creating Services
• • • • • • •
Possibly Optimize Imagery Do NOT Reproject or sample data (will be done OTF) Recompute footprints to remove NoData Use Calculate to Set MinPS and MaxPS Use DispOrder or SysOrder to force order Consider location of processes – Use Edit Raster Properties Consider adding attributes to define accuracy
Demo
• Create Image Service Definition from – GTOP – SRTM – NED 30m – NED 10m – Rasterized Lidar Data
Creating Derived Services
• • • •
Enable Multiple Service from same source Enables rendering: EG for WMS Enables high compression for transmission Simplifies data management
• Use SaveAs, or reference using ISRefs • Developer can use client defined processes
Using Multiple Services • Height = (First Return – Bare Earth)
• Correct Geoid
GeoProcessing
• In ArcGIS Desktop – Export AOI at required resolution
• In Geoprocessing Tools – Use Make Image Server Layer
9.3.1 Limitations – With Image Services • Transparency of No Data not handled well – Use footprints
• • • •
Black lines at edges of export with focal processes Source Vertical Datum Different renderer then ArcGIS – Solution: Use as Layer on ArcGIS Server
ArcGIS 10 – Improvements for Imagery and Elevation
• Management – Mosaic Dataset – Stored in GeoDatabase
• Access – More formats
• Improved No Data Support • Image Analysis Window – Access OTF processing in Desktop
• Spatial Analysis – Direct use of multiple formats – Improved speed
ArcGIS The Platform for Fully Integrated GIS and Imagery • • • •
Integrating Imagery as core to GIS Management, Dissemination, Visualization and Analysis Solution for wide range of user requirements Maximizes the value of imagery including Imagery
• • •
Single Source for all elevation Multiple Products from a single source Maintainability
GIS
++
Learn More http://www.esri.com/training
• Instructor-Led Training – Introduction to ArcGIS Server Image Extension – ArcGIS Server: Web Administration Using the Microsoft .NET Framework
• Free Web Training Seminar – Serving and Managing Imagery with ArcGIS
ESRI Training…keep critical skills up to date
