Using the GeoEvent Processor for Server
Using the GeoEvent Processor for Server Matthias Schenker, Esri Switzerland Javier Abadia, Esri España
Geodata comes alive…
… in Realtime
Realtime Data Features Police Car Police Person Ambulance
Network Sensor
Applications
Warehouse Item
Mobile Desktop Web Runtime
Storm Wind
46°
Temperature Earthquake Wild Fire
“Sensors in Motion” and or “Values in Motion”
Realtime GIS • Continuous
stream of events • Sensors (1…n) as Eventsource • Event = Current state of the sensor
Are my field personnel working within the designated area?
Continuous Analysis
Features Equipment
Outside Boundary
Alert
Applications Mobile Desktop Web Runtime
Is everything under control?
Continuous Analysis
Sensors Gauge
Outside Range
Alert
Application Mobile Desktop Web Runtime
We will be with you in 15 minutes…
Continuous Analysis
Features Delivery Vehicle
Enter Customer Area
Notification
Application Mobile Desktop Web Runtime
Challenges • How
does data get into the system? • Analysis: How/Where is data converted into information? • How does the information get to the user?
GeoEvent Processor Basics Matthias Schenker
ArcGIS GeoEvent Processor for Server
Real-Time Data
Applications
ArcGIS Server
Features
GeoEvent Processor Alerts
Mobile Desktop Web Runtime
im ws://
Continuous Processing and Analysis Connectors
Connectors
Receiving real-time data ArcGIS for Server GeoEvent Processor Real-Time Data
Receive from a Socket Receive Tweets
ws://
Receive Instagram
http:// Receive on a REST Endpoint
Receive from a Web Socket Receive Air Traffic Control
Receive Trimble
Receive RSS
CoT
Receive Cursor-on-Target
Receive Sierra Wireless
Watch a folder for new Files
VMF
Receive VMF
NMEA Receive NMEA GPX Receive GPX
Poll an ArcGIS Server
Geo Message
http:// Poll an external website
Receive Geomessages Receive Vessels
You can add your own connectors http://www.arcgis.com/home/group.html?owner=GeoEventTeam&title=ArcG IS%20GeoEvent%20Processor
Sending real-time data ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im
Real-Time Data
ws:// Update a Feature
You can add your own connectors
im
Add a Feature
Send a Tweet
Send an email
Send to RabbitMQ
Send an Instant Message
Send to ActiveMQ
Send a Text Message
Send to HornetQ
Publish to REST Endpoint
Write to Hadoop
ws:// Send to a Web Socket Send to a Socket Write to a File
Write to MongoDB Send to CESIUM
Sending real-time data to features Dashboard Operations Dashboard for ArcGIS
Your Applications Web APIs
ArcGIS for Server GeoEvent Processor
ArcGIS Online / Portal
JavaScript
Operation View
Silverlight/WPF
Web Map
GeoEvent Services
Mobile GIS Output
Input
Real-Time Data
Flex
iOS Feature Layer
Android Windows Phone
Filters
Processors
Runtime Update a Feature Add a Feature
Windows Store Java Mac OS X Qt
Real-time data is often “Big Data” ArcGIS for Desktop Geoprocessing GIS Tools for Hadoop
ArcGIS for Server GeoEvent Processor
Input
Real-time data
Output
GeoEvent Services
Filters
2013/03/06/10/vehicles-2013-03-06-10.tsv 2013/03/06/11/vehicles-2013-03-06-11.tsv 2013/03/06/12/vehicles-2013-03-06-12.tsv 2013/03/06/13/vehicles-2013-03-06-13.tsv
2013/03/06/10/vehicles-2013-03-06-10.json 2013/03/06/11/vehicles-2013-03-06-11.json 2013/03/06/12/vehicles-2013-03-06-12.json 2013/03/06/13/vehicles-2013-03-06-13.json
Processors
Write to Hadoop Write to MongoDB
GIS Tools for Hadoop http://github.com/Esri/gis-tools-for-hadoop
Filters and Processes Javier Abadía
Continuous processing and analysis ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im Real-Time Data ws://
Twitter Monitor (GeoEvent Service) Twitter Input
Geolocated Filter
Add Tweet Feature Write Tweets to Hadoop
A GeoEvent Service configures the flow of (geo-)events: Input - Output - Filters and processes -
Continuous processing and analysis ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im Echtzeit-Daten ws://
Air Traffic Monitor (GeoEvent Service) Update Flight Feature GEOMETRY inside RestrictedArea/*
Flight Input
Add Violation Feature
Altitude < 10,000 GEOMETRY enter RestrictedArea/*
Instant Message Out Send an email
-
Input
-
Output
-
Filters and processes
Processors • • •
Calculate new fields on a geoevent Modify a geoevent Derive a new geoevent ArcGIS Server
with GeoEvent Processor
GeoEvent Service GeoTagger
Field Calculator
Field Enricher
Input 1
Output 1 Field Reducer
Field Mapper
Incident Detector
Incident Detector processor •
creates an incident upon an opening condition being met,
•
maintains state for the duration of the incident,
•
closes the incident based on a closing condition or expiration.
GeoEvent Service
66
68
73
Output 1
Incident Detector
Input 1
81
92
90
91
90
68
65
noiseDecibel >= 90 Opening Condition
81
92
Close Incident
90
91
90 Open Incident
DEMO
Ship Tracking
Custom Connectors and REST API Matthias Schenker
Geoevent Processor SDK
•
Included with the Setup
•
Contains
•
-
Library files
-
Samples
-
Javadoc API reference
-
Developer Guide
Build -
Processors and Filters
-
Adapters
-
Transports
Processors and Filters
Transport and Adapter
• Transport: Generate raw Events in their native format • Adapter: Generate Geoevents out of raw events
REST Admin API
https://[...]/geoevent/admin/outputs/ https://[...]/geoevent/rest/outputs/
CRUD Read-Only
Operations
• Use
e.g. Poster in Firefox
Receive Input /get Output via REST
•
GeoEvent Receiver http://[...]/geoevent/rest/receiver/
•
Get output via REST from output connector
•
GeoEvent Cache http://[...]/geoevent/rest/cache/
Real-Time Applications Javier Abadía
Maps
Counters
Gauges
Operations Dashboard
Dynamic Data Sources = Automatic Updates
DEMO
Real Time Dashboard
ArcGIS.com viewer
new Refresh Interval parameter
Javascript API
setRefreshInterval() (v3.7) StreamLayer (v3.6)
WebSocket
StreamLayer
Your Applications Web APIs JavaScript
ArcGIS for Server
Flex Silverlight/WPF
GeoEvent Processor GeoEvent Services Output
Input
Real-Time Data
Mobile GIS iOS Feature Layer
Android Windows Phone
Filters
Processors
Runtime Update a Feature Add a Feature
Windows Store Java Mac OS X
ws:// Send to a Web Socket
Javascript API
Qt
setRefreshInterval() (v3.7) StreamLayer (v3.6)
Geodata comes alive…
… in Realtime
Realtime Data Features Police Car Police Person Ambulance
Network Sensor
Applications
Warehouse Item
Mobile Desktop Web Runtime
Storm Wind
46°
Temperature Earthquake Wild Fire
“Sensors in Motion” and or “Values in Motion”
Realtime GIS • Continuous
stream of events • Sensors (1…n) as Eventsource • Event = Current state of the sensor
Are my field personnel working within the designated area?
Continuous Analysis
Features Equipment
Outside Boundary
Alert
Applications Mobile Desktop Web Runtime
Is everything under control?
Continuous Analysis
Sensors Gauge
Outside Range
Alert
Application Mobile Desktop Web Runtime
We will be with you in 15 minutes…
Continuous Analysis
Features Delivery Vehicle
Enter Customer Area
Notification
Application Mobile Desktop Web Runtime
Challenges • How
does data get into the system? • Analysis: How/Where is data converted into information? • How does the information get to the user?
GeoEvent Processor Basics Matthias Schenker
ArcGIS GeoEvent Processor for Server
Real-Time Data
Applications
ArcGIS Server
Features
GeoEvent Processor Alerts
Mobile Desktop Web Runtime
im ws://
Continuous Processing and Analysis Connectors
Connectors
Receiving real-time data ArcGIS for Server GeoEvent Processor Real-Time Data
Receive from a Socket Receive Tweets
ws://
Receive Instagram
http:// Receive on a REST Endpoint
Receive from a Web Socket Receive Air Traffic Control
Receive Trimble
Receive RSS
CoT
Receive Cursor-on-Target
Receive Sierra Wireless
Watch a folder for new Files
VMF
Receive VMF
NMEA Receive NMEA GPX Receive GPX
Poll an ArcGIS Server
Geo Message
http:// Poll an external website
Receive Geomessages Receive Vessels
You can add your own connectors http://www.arcgis.com/home/group.html?owner=GeoEventTeam&title=ArcG IS%20GeoEvent%20Processor
Sending real-time data ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im
Real-Time Data
ws:// Update a Feature
You can add your own connectors
im
Add a Feature
Send a Tweet
Send an email
Send to RabbitMQ
Send an Instant Message
Send to ActiveMQ
Send a Text Message
Send to HornetQ
Publish to REST Endpoint
Write to Hadoop
ws:// Send to a Web Socket Send to a Socket Write to a File
Write to MongoDB Send to CESIUM
Sending real-time data to features Dashboard Operations Dashboard for ArcGIS
Your Applications Web APIs
ArcGIS for Server GeoEvent Processor
ArcGIS Online / Portal
JavaScript
Operation View
Silverlight/WPF
Web Map
GeoEvent Services
Mobile GIS Output
Input
Real-Time Data
Flex
iOS Feature Layer
Android Windows Phone
Filters
Processors
Runtime Update a Feature Add a Feature
Windows Store Java Mac OS X Qt
Real-time data is often “Big Data” ArcGIS for Desktop Geoprocessing GIS Tools for Hadoop
ArcGIS for Server GeoEvent Processor
Input
Real-time data
Output
GeoEvent Services
Filters
2013/03/06/10/vehicles-2013-03-06-10.tsv 2013/03/06/11/vehicles-2013-03-06-11.tsv 2013/03/06/12/vehicles-2013-03-06-12.tsv 2013/03/06/13/vehicles-2013-03-06-13.tsv
2013/03/06/10/vehicles-2013-03-06-10.json 2013/03/06/11/vehicles-2013-03-06-11.json 2013/03/06/12/vehicles-2013-03-06-12.json 2013/03/06/13/vehicles-2013-03-06-13.json
Processors
Write to Hadoop Write to MongoDB
GIS Tools for Hadoop http://github.com/Esri/gis-tools-for-hadoop
Filters and Processes Javier Abadía
Continuous processing and analysis ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im Real-Time Data ws://
Twitter Monitor (GeoEvent Service) Twitter Input
Geolocated Filter
Add Tweet Feature Write Tweets to Hadoop
A GeoEvent Service configures the flow of (geo-)events: Input - Output - Filters and processes -
Continuous processing and analysis ArcGIS for Server
Mobile Desktop Web Runtime
GeoEvent Processor im Echtzeit-Daten ws://
Air Traffic Monitor (GeoEvent Service) Update Flight Feature GEOMETRY inside RestrictedArea/*
Flight Input
Add Violation Feature
Altitude < 10,000 GEOMETRY enter RestrictedArea/*
Instant Message Out Send an email
-
Input
-
Output
-
Filters and processes
Processors • • •
Calculate new fields on a geoevent Modify a geoevent Derive a new geoevent ArcGIS Server
with GeoEvent Processor
GeoEvent Service GeoTagger
Field Calculator
Field Enricher
Input 1
Output 1 Field Reducer
Field Mapper
Incident Detector
Incident Detector processor •
creates an incident upon an opening condition being met,
•
maintains state for the duration of the incident,
•
closes the incident based on a closing condition or expiration.
GeoEvent Service
66
68
73
Output 1
Incident Detector
Input 1
81
92
90
91
90
68
65
noiseDecibel >= 90 Opening Condition
81
92
Close Incident
90
91
90 Open Incident
DEMO
Ship Tracking
Custom Connectors and REST API Matthias Schenker
Geoevent Processor SDK
•
Included with the Setup
•
Contains
•
-
Library files
-
Samples
-
Javadoc API reference
-
Developer Guide
Build -
Processors and Filters
-
Adapters
-
Transports
Processors and Filters
Transport and Adapter
• Transport: Generate raw Events in their native format • Adapter: Generate Geoevents out of raw events
REST Admin API
https://[...]/geoevent/admin/outputs/ https://[...]/geoevent/rest/outputs/
CRUD Read-Only
Operations
• Use
e.g. Poster in Firefox
Receive Input /get Output via REST
•
GeoEvent Receiver http://[...]/geoevent/rest/receiver/
•
Get output via REST from output connector
•
GeoEvent Cache http://[...]/geoevent/rest/cache/
Real-Time Applications Javier Abadía
Maps
Counters
Gauges
Operations Dashboard
Dynamic Data Sources = Automatic Updates
DEMO
Real Time Dashboard
ArcGIS.com viewer
new Refresh Interval parameter
Javascript API
setRefreshInterval() (v3.7) StreamLayer (v3.6)
WebSocket
StreamLayer
Your Applications Web APIs JavaScript
ArcGIS for Server
Flex Silverlight/WPF
GeoEvent Processor GeoEvent Services Output
Input
Real-Time Data
Mobile GIS iOS Feature Layer
Android Windows Phone
Filters
Processors
Runtime Update a Feature Add a Feature
Windows Store Java Mac OS X
ws:// Send to a Web Socket
Javascript API
Qt
setRefreshInterval() (v3.7) StreamLayer (v3.6)
