What is a Mosaic Dataset - CiteSeerX
Esri International User Conference San Diego, California
Technical Workshops |July 25, 2012
Managing Imagery and Raster Data Using Mosaic Datasets Hong Xu, Prashant Mangtani
Presentation Overview
•
Introduction to mosaic dataset
•
Building mosaic datasets
•
Design mosaic datasets for your organization
•
Maintain and update
•
Migrate to mosaic datasets
•
Q&A
Challenges of Managing Imagery •
Many image sources -
Aerial photographs
-
Satellite imagery from many sensors
-
DEM and scanned maps
-
Analytical data
•
High resolution and large volume
•
Need a data model -
Store efficiently
-
Search easily
-
Process fast
-
Be able to accessible to multiple clients
What is a Mosaic Dataset •
A geodatabase model designed for managing collections of images -
•
•
Catalog, process, search, display, and update
Stored as tables in geodatabase -
Footprint table references source images
-
Boundary, raster type
-
Seamlines, color correction etc…
Displayed as a composite layer in Map -
Boundary
-
Footprint
-
Image
Mosaic Dataset
Advantages of Mosaic Datasets •
•
Dynamic mosaicking -
Seamless display at all levels
-
Overviews to speed up display
Processing images on-the-fly -
Using raster function
-
Reduce process time
•
Manage large image collections
•
Fast to create
•
Support images of multiple sensor platforms
•
Streamline update and maintain quality
Use Mosaic Dataset as a Catalog •
Selection and query -
Add selected rasters to Map
-
Lock the selected rasters
•
View raster and metadata
•
Time aware
Use Mosaic Dataset as an Image •
Is a raster dataset
•
Provide a seamless mosaic on the fly based on the mosaic rule -
-
Closest to the center
Mosaic method to sort the rasters -
Closest to center
-
By attribute
-
Closest to nadir
-
North west (default)
-
Seamline
Mosaic operator to resolve the overlaps -
First/Min/Max/Mean/Blend By attribute: cloud cover
•
Export or clip to a raster dataset
•
Use as an input to geoprocessing tool
Share Mosaic Dataset as an Image Service •
Provide mosaic dataset functionality to web users
•
Access as a catalog -
Select/download selected images
-
Time aware
•
Access as a seamless mosaic
•
REST, Web API, WCS, WMS
•
Require image extension license
•
Session: Publishing image services in ArcGIS (Room 28D) -
Thursday 08:30 AM - 09:45 AM
-
Thursday 03:15 PM - 04:30 PM
Demo: Mosaic dataset basics
Building Mosaic Datasets
•
Cataloging and searching properties
•
Make mosaic dataset look good - seamless mosaic
Basic Workflow
•
Create a mosaic dataset
•
Add rasters -
Build footprint
-
Build boundary
•
Build overviews (optional)
•
Mosaic dataset toolset
•
Automate with model and python
Creating a Mosaic Dataset Create Mosaic Dataset tool - Create footprint table schema OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
•
Footprint table – internal raster catalog with more fields
•
Parameters to set -
Define spatial reference -
Web Mercator or spatial reference used by your organization
-
Number of bands and pixel types
-
Product definition -
Used for automatic band matching
-
Examples: RGB, RGBI, Landset, etc
…
Adding Rasters Add Rasters to Mosaic Dataset tool - Populate field values
•
•
OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
…
Raster value contains a raster function (process definition) -
References images on disk
-
Process on the fly
-
Contains metadata, statistics, thumbnails
A raster type defines what to add -
Understand sensor metadata
-
Construct the raster function
-
Add additional fields -
Sensor, CloudCover, etc.
Supported Raster Types Define data types can be managed by mosaic dataset •
Raster dataset -
•
TIFF , Mrsid, JP2000, NITF, etc.
Satellite images Landsat 7 ETM+, Landsat 4-5 TM, Landsat 1-5 MSS - QuickBird, IKONOS, GeoEye-1, WorldView I and II - RapidEye, SPOT, Radarsat 2, FORMOSAT-2, KOMPSAT-2 -
•
Aerial images -
Match-AT, ISAT, Applanix
Supported Raster Types (Continue) Define data types can be managed by mosaic dataset •
Lidar and Terrain -
•
Military Formats -
•
CIB, CADRG, DTED, HRE
Services -
•
LAS, LAS Dataset, and Terrain
Image service, cache map service, WCS, WMS
Table -
Add existing mosaic datasets by copying all the rows
Tips for Adding Rasters •
Use DEM orthorectifying sensor data when possible
•
Use Gram-Schmidt pansharpen method
•
Statistics -
Do not calculate for processed data
-
Calculate for sensor data
•
Build pyramids
•
Adding from folder when possible -
Required by synchronizing a mosaic dataset
•
Define datum transform if needed (GP environment)
•
Handle invalid spatial reference -
Define for missing data
-
Overwrite wrong spatial reference
Calculate Mosaic Dataset Item Visibility Using Calculate Cell Size Ranges tool OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
5
Polygon
filename1.tif
90
270
90
90
Overviews
•
Mosaic dataset contains rasters of different resolutions
•
MinPS and MaxPS define the visibility of these rasters
•
Calculated based on sources and overlaps -
•
Default cell size range factor is 10
Can be set using field calculator 30m 5m
visible range 30-300 visible range 0-50 à 5-30
1m
visible range 0-5
…
Demo: Mosaic dataset cataloging and search
Topics for Building Mosaic Datasets
•
Cataloging and searching
•
Make mosaic dataset look good – seamless mosaic
Refine Processing – Raster Functions Edit raster function for mosaic items and mosaic dataset
•
Support many functions
•
Edit one item or the mosaic dataset -
•
Edit multiple items -
•
using raster function editor using raster function batch editor
The new Edit Raster Function tool -
For editing item (s) or mosaic dataset
Handling the Image Background Use Build Footprints and Set NoData tools
•
Build Footprint tool -
By radiometry
-
By geometry
•
Edit using Editor
•
Import Mosaic Dataset Geometry tool
•
Define NoData GP tool -
Based on a value
-
Based on a range
Use footprint
Use NoData
Build Boundary
•
Define the boundary of the mosaic dataset -
•
Pixels outside the boundary will be clipped
Build using Build Boundary tool -
Calculated based on footprints
•
Can be modified using Editor
•
Import Mosaic Dataset Geometry tool
Statistics in Mosaic Datasets Use Build Pyramids and Statistics tool •
Statistics in multiple places -
Source raster dataset
-
Mosaic dataset item (function raster)
-
Mosaic dataset
Source raster
•
Pyramids exist in source dataset
•
Calculate Statistics tool -
•
Function raster
for mosaic dataset statistics
Build Pyramids and Statistics Tool -
For statistics of items and source raster dataset
Mosaic dataset
Color Correcting Mosaic Dataset Use Color Balance Mosaic Dataset tool or Color Correction window
•
Based on a calculated color surface -
Single color
-
Color grid
-
First, second, third polynomial
•
Based on an existing target raster
•
Support excluded area
Before
After
Building Seamlines Use Build Seamlines tool •
Used in mosaiking rasters in a mosaic dataset
•
Stored in the seamline table
•
Used by Seamline mosaic method -
•
•
Blend along the seamlines with a default width
Generate seamlines automatically -
Radiometry
-
Geometry
-
Etc.
Edit and import seamlines
1
2
Building Overviews Cell size=100
Using Build Overviews tool
OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
3
Polygon
filename1.tif
90
900
90
90
Overviews
4
Polygon
filename2.tif
90
900
90
90
Overviews
•
Resampled rasters referenced in the footprint table
•
Calculated based on default -
TIFF with JPEG compression
-
Size of 5120x5120 and factor of 3
-
Use Define Overview tool to refine the parameters
•
Used for fast display at all levels
•
Add a raster dataset or a service as an overview
…
Analyzing Mosaic Datasets Using Analyze Mosaic Dataset tool
•
Recommended for publishing as image services
•
Check raster
•
Check footprint geometry
•
Check function chain
•
Check broken path
•
Performance
•
etc.
Updating and Synchronizing Mosaic Dataset Using Synchronize Mosaic Dataset GP tool
•
Data must be added from folder mosaic datasets
•
Update existing items -
Any change in source images
-
Any change in rasters
-
Option to maintain the existing fields
•
Add new rasters
•
Remove items of broken source
Moving Mosaic Dataset •
•
Move all -
Copy the FGDB where the mosaic dataset resides
-
Copy the source and overview images
-
Use Repair dialog to repair the paths
-
Or Repair Mosaic Dataset Paths GP tool
Extract a portion -
Use Distributed Geodatabase toolbar
-
Create a new folder
-
Create a target File geodatabase
-
Run Extract
-
Copy/move the whole folder
Demo: Make mosaic look good
Design mosaic datasets within your organization
Source and Master Mosaic Datasets
•
Source mosaic dataset -
•
Master mosaic dataset -
•
Create from adding source images Create from source mosaic datasets -
Table type by copying rows
-
Raster Dataset type by adding the whole mosaic dataset
Support update mechanism -
Synchronize source mosaic if source images updated
-
Synchronize master mosaic if source mosaic is updated
Reference Mosaic Dataset •
References an external mosaic dataset or raster catalog
•
Created by specifying Definition query - Area of interest -
Boundary = My County
Where Sensor = Landsat and Cloud
Technical Workshops |July 25, 2012
Managing Imagery and Raster Data Using Mosaic Datasets Hong Xu, Prashant Mangtani
Presentation Overview
•
Introduction to mosaic dataset
•
Building mosaic datasets
•
Design mosaic datasets for your organization
•
Maintain and update
•
Migrate to mosaic datasets
•
Q&A
Challenges of Managing Imagery •
Many image sources -
Aerial photographs
-
Satellite imagery from many sensors
-
DEM and scanned maps
-
Analytical data
•
High resolution and large volume
•
Need a data model -
Store efficiently
-
Search easily
-
Process fast
-
Be able to accessible to multiple clients
What is a Mosaic Dataset •
A geodatabase model designed for managing collections of images -
•
•
Catalog, process, search, display, and update
Stored as tables in geodatabase -
Footprint table references source images
-
Boundary, raster type
-
Seamlines, color correction etc…
Displayed as a composite layer in Map -
Boundary
-
Footprint
-
Image
Mosaic Dataset
Advantages of Mosaic Datasets •
•
Dynamic mosaicking -
Seamless display at all levels
-
Overviews to speed up display
Processing images on-the-fly -
Using raster function
-
Reduce process time
•
Manage large image collections
•
Fast to create
•
Support images of multiple sensor platforms
•
Streamline update and maintain quality
Use Mosaic Dataset as a Catalog •
Selection and query -
Add selected rasters to Map
-
Lock the selected rasters
•
View raster and metadata
•
Time aware
Use Mosaic Dataset as an Image •
Is a raster dataset
•
Provide a seamless mosaic on the fly based on the mosaic rule -
-
Closest to the center
Mosaic method to sort the rasters -
Closest to center
-
By attribute
-
Closest to nadir
-
North west (default)
-
Seamline
Mosaic operator to resolve the overlaps -
First/Min/Max/Mean/Blend By attribute: cloud cover
•
Export or clip to a raster dataset
•
Use as an input to geoprocessing tool
Share Mosaic Dataset as an Image Service •
Provide mosaic dataset functionality to web users
•
Access as a catalog -
Select/download selected images
-
Time aware
•
Access as a seamless mosaic
•
REST, Web API, WCS, WMS
•
Require image extension license
•
Session: Publishing image services in ArcGIS (Room 28D) -
Thursday 08:30 AM - 09:45 AM
-
Thursday 03:15 PM - 04:30 PM
Demo: Mosaic dataset basics
Building Mosaic Datasets
•
Cataloging and searching properties
•
Make mosaic dataset look good - seamless mosaic
Basic Workflow
•
Create a mosaic dataset
•
Add rasters -
Build footprint
-
Build boundary
•
Build overviews (optional)
•
Mosaic dataset toolset
•
Automate with model and python
Creating a Mosaic Dataset Create Mosaic Dataset tool - Create footprint table schema OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
•
Footprint table – internal raster catalog with more fields
•
Parameters to set -
Define spatial reference -
Web Mercator or spatial reference used by your organization
-
Number of bands and pixel types
-
Product definition -
Used for automatic band matching
-
Examples: RGB, RGBI, Landset, etc
…
Adding Rasters Add Rasters to Mosaic Dataset tool - Populate field values
•
•
OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
…
Raster value contains a raster function (process definition) -
References images on disk
-
Process on the fly
-
Contains metadata, statistics, thumbnails
A raster type defines what to add -
Understand sensor metadata
-
Construct the raster function
-
Add additional fields -
Sensor, CloudCover, etc.
Supported Raster Types Define data types can be managed by mosaic dataset •
Raster dataset -
•
TIFF , Mrsid, JP2000, NITF, etc.
Satellite images Landsat 7 ETM+, Landsat 4-5 TM, Landsat 1-5 MSS - QuickBird, IKONOS, GeoEye-1, WorldView I and II - RapidEye, SPOT, Radarsat 2, FORMOSAT-2, KOMPSAT-2 -
•
Aerial images -
Match-AT, ISAT, Applanix
Supported Raster Types (Continue) Define data types can be managed by mosaic dataset •
Lidar and Terrain -
•
Military Formats -
•
CIB, CADRG, DTED, HRE
Services -
•
LAS, LAS Dataset, and Terrain
Image service, cache map service, WCS, WMS
Table -
Add existing mosaic datasets by copying all the rows
Tips for Adding Rasters •
Use DEM orthorectifying sensor data when possible
•
Use Gram-Schmidt pansharpen method
•
Statistics -
Do not calculate for processed data
-
Calculate for sensor data
•
Build pyramids
•
Adding from folder when possible -
Required by synchronizing a mosaic dataset
•
Define datum transform if needed (GP environment)
•
Handle invalid spatial reference -
Define for missing data
-
Overwrite wrong spatial reference
Calculate Mosaic Dataset Item Visibility Using Calculate Cell Size Ranges tool OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
5
Polygon
filename1.tif
90
270
90
90
Overviews
•
Mosaic dataset contains rasters of different resolutions
•
MinPS and MaxPS define the visibility of these rasters
•
Calculated based on sources and overlaps -
•
Default cell size range factor is 10
Can be set using field calculator 30m 5m
visible range 30-300 visible range 0-50 à 5-30
1m
visible range 0-5
…
Demo: Mosaic dataset cataloging and search
Topics for Building Mosaic Datasets
•
Cataloging and searching
•
Make mosaic dataset look good – seamless mosaic
Refine Processing – Raster Functions Edit raster function for mosaic items and mosaic dataset
•
Support many functions
•
Edit one item or the mosaic dataset -
•
Edit multiple items -
•
using raster function editor using raster function batch editor
The new Edit Raster Function tool -
For editing item (s) or mosaic dataset
Handling the Image Background Use Build Footprints and Set NoData tools
•
Build Footprint tool -
By radiometry
-
By geometry
•
Edit using Editor
•
Import Mosaic Dataset Geometry tool
•
Define NoData GP tool -
Based on a value
-
Based on a range
Use footprint
Use NoData
Build Boundary
•
Define the boundary of the mosaic dataset -
•
Pixels outside the boundary will be clipped
Build using Build Boundary tool -
Calculated based on footprints
•
Can be modified using Editor
•
Import Mosaic Dataset Geometry tool
Statistics in Mosaic Datasets Use Build Pyramids and Statistics tool •
Statistics in multiple places -
Source raster dataset
-
Mosaic dataset item (function raster)
-
Mosaic dataset
Source raster
•
Pyramids exist in source dataset
•
Calculate Statistics tool -
•
Function raster
for mosaic dataset statistics
Build Pyramids and Statistics Tool -
For statistics of items and source raster dataset
Mosaic dataset
Color Correcting Mosaic Dataset Use Color Balance Mosaic Dataset tool or Color Correction window
•
Based on a calculated color surface -
Single color
-
Color grid
-
First, second, third polynomial
•
Based on an existing target raster
•
Support excluded area
Before
After
Building Seamlines Use Build Seamlines tool •
Used in mosaiking rasters in a mosaic dataset
•
Stored in the seamline table
•
Used by Seamline mosaic method -
•
•
Blend along the seamlines with a default width
Generate seamlines automatically -
Radiometry
-
Geometry
-
Etc.
Edit and import seamlines
1
2
Building Overviews Cell size=100
Using Build Overviews tool
OID
Shape
Raster
Name
MinPS
MaxPS
LowPS
HighPS
Category
1
Polygon
P01.met
0
90
10
30
Primary
2
Polygon
P02.met
0
90
10
30
Primary
3
Polygon
filename1.tif
90
900
90
90
Overviews
4
Polygon
filename2.tif
90
900
90
90
Overviews
•
Resampled rasters referenced in the footprint table
•
Calculated based on default -
TIFF with JPEG compression
-
Size of 5120x5120 and factor of 3
-
Use Define Overview tool to refine the parameters
•
Used for fast display at all levels
•
Add a raster dataset or a service as an overview
…
Analyzing Mosaic Datasets Using Analyze Mosaic Dataset tool
•
Recommended for publishing as image services
•
Check raster
•
Check footprint geometry
•
Check function chain
•
Check broken path
•
Performance
•
etc.
Updating and Synchronizing Mosaic Dataset Using Synchronize Mosaic Dataset GP tool
•
Data must be added from folder mosaic datasets
•
Update existing items -
Any change in source images
-
Any change in rasters
-
Option to maintain the existing fields
•
Add new rasters
•
Remove items of broken source
Moving Mosaic Dataset •
•
Move all -
Copy the FGDB where the mosaic dataset resides
-
Copy the source and overview images
-
Use Repair dialog to repair the paths
-
Or Repair Mosaic Dataset Paths GP tool
Extract a portion -
Use Distributed Geodatabase toolbar
-
Create a new folder
-
Create a target File geodatabase
-
Run Extract
-
Copy/move the whole folder
Demo: Make mosaic look good
Design mosaic datasets within your organization
Source and Master Mosaic Datasets
•
Source mosaic dataset -
•
Master mosaic dataset -
•
Create from adding source images Create from source mosaic datasets -
Table type by copying rows
-
Raster Dataset type by adding the whole mosaic dataset
Support update mechanism -
Synchronize source mosaic if source images updated
-
Synchronize master mosaic if source mosaic is updated
Reference Mosaic Dataset •
References an external mosaic dataset or raster catalog
•
Created by specifying Definition query - Area of interest -
Boundary = My County
Where Sensor = Landsat and Cloud
