Real-Time Visual Analytics for Event Data Streams - Semantic Scholar
27th March 2012, ACM SAC 2012 Riva del Garda (Trento), Italy
Real-Time Visual Analytics for Event Data Streams Fabian Fischer, Florian Mansmann, Daniel A. Keim Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
Massive Processing Power Burst Detection
Classification
Statistics
Machine Learning
Clustering
Data Mining
Human Analyst Expert Knowledge
Cognition
Intuition
Experience Understanding
…
… Interactive Visualization
is a way to tightly combine human factors and data analysis.
Visual Analytics Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
2
Use Case for Event Streams
Analyzing System Log Events (event stream of server log messages)
The National Archives (UK), 2011
Framework Architecture Real-Time Visual Analytics for Event Data Streams
Data Streams
Event Service
analyzed events
Event EventAnalyzer(s) Analyzer(s) Visualizer Event Event Analyzer(s)
raw messages
Message Broker connect to data storage
raw messages
Normalization Fingerprint
EventAnalyzer(s) Analyzer(s) Event Analyzer(s) Event Event Analyzer(s)
Rules Scoring
analyzed events
Aggregation
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
Data Storage 4
Relaxed Event Timeline Visualization Focus on Temporal Aspect of Data Streams (Monitoring & Exploration)
selected scale: one hour (h)
color mapped to priority
s1
A
s2 s3
K
B C
E D
F
J G
H
hnow - 1
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
I hnow
6
Demo/Video
Main Contributions • Generic processing and analysis architecture for event data streams to support real-time visual analytics applications. • A system for pluggable visualizations for real-time and historical event data. • Dynamic timeline visualization to directly interact with multiple streams to visualize highly co-occurring events. Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
12
Future Work • Controlled system evaluation. • Integration of advanced algorithms for burst and anomaly detection. • Integration of more visualizations based on the learned design principles. • Use the Event Visualizer for other datasets. – Feb 2012 – Successful participation in the Honeynet Forensic Challenge 2011/10 [1]. [1] http://ff.cx/fc10/ Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
13
Thank you very much for your attention! Questions? For more information about this work or about visual analytics please contact Fabian Fischer Tel. +49 7531 88-2780 [email protected]
@f2cx
http://ff.cx/
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
14
References I
J. Thomas and K. Cook (2005). Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Computer Society, 2005.
W. Aigner, S. Miksch, H. Schumann, and C. Tominski (2011). Visualization of Time-Oriented Data. Human-Computer Interaction. Springer Verlag, 1st edition, 2011.
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
15
References II
G. Chin, M. Singhal, G. Nakamura, V. Gurumoorthi, and N. Freeman-Cadoret (2009). Visual Analysis of Dynamic Data Streams. Information Visualization, 8(3):212-229, 2009.
M. Schaefer, F. Wanner, F. Mansmann, C. Scheible, V. Stennett, A. T. Hasselrot, and D. A. Keim (2011). Visual Pattern Discovery in Timed Event Data. In Proceedings of Conference on Visualization and Data Analysis, 2011.
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
16
Real-Time Visual Analytics for Event Data Streams Fabian Fischer, Florian Mansmann, Daniel A. Keim Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
Massive Processing Power Burst Detection
Classification
Statistics
Machine Learning
Clustering
Data Mining
Human Analyst Expert Knowledge
Cognition
Intuition
Experience Understanding
…
… Interactive Visualization
is a way to tightly combine human factors and data analysis.
Visual Analytics Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
2
Use Case for Event Streams
Analyzing System Log Events (event stream of server log messages)
The National Archives (UK), 2011
Framework Architecture Real-Time Visual Analytics for Event Data Streams
Data Streams
Event Service
analyzed events
Event EventAnalyzer(s) Analyzer(s) Visualizer Event Event Analyzer(s)
raw messages
Message Broker connect to data storage
raw messages
Normalization Fingerprint
EventAnalyzer(s) Analyzer(s) Event Analyzer(s) Event Event Analyzer(s)
Rules Scoring
analyzed events
Aggregation
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
Data Storage 4
Relaxed Event Timeline Visualization Focus on Temporal Aspect of Data Streams (Monitoring & Exploration)
selected scale: one hour (h)
color mapped to priority
s1
A
s2 s3
K
B C
E D
F
J G
H
hnow - 1
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
I hnow
6
Demo/Video
Main Contributions • Generic processing and analysis architecture for event data streams to support real-time visual analytics applications. • A system for pluggable visualizations for real-time and historical event data. • Dynamic timeline visualization to directly interact with multiple streams to visualize highly co-occurring events. Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
12
Future Work • Controlled system evaluation. • Integration of advanced algorithms for burst and anomaly detection. • Integration of more visualizations based on the learned design principles. • Use the Event Visualizer for other datasets. – Feb 2012 – Successful participation in the Honeynet Forensic Challenge 2011/10 [1]. [1] http://ff.cx/fc10/ Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
13
Thank you very much for your attention! Questions? For more information about this work or about visual analytics please contact Fabian Fischer Tel. +49 7531 88-2780 [email protected]
@f2cx
http://ff.cx/
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
14
References I
J. Thomas and K. Cook (2005). Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Computer Society, 2005.
W. Aigner, S. Miksch, H. Schumann, and C. Tominski (2011). Visualization of Time-Oriented Data. Human-Computer Interaction. Springer Verlag, 1st edition, 2011.
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
15
References II
G. Chin, M. Singhal, G. Nakamura, V. Gurumoorthi, and N. Freeman-Cadoret (2009). Visual Analysis of Dynamic Data Streams. Information Visualization, 8(3):212-229, 2009.
M. Schaefer, F. Wanner, F. Mansmann, C. Scheible, V. Stennett, A. T. Hasselrot, and D. A. Keim (2011). Visual Pattern Discovery in Timed Event Data. In Proceedings of Conference on Visualization and Data Analysis, 2011.
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
16
