Markov Modeling of Time-Series Data via Spectral Analysis
1st International Conference on InfoSymbiotics / DDDAS Session-2: Process Monitoring
Markov Modeling of Time-Series Data via Spectral Analysis Nurali Virani Department of Mechanical Engineering The Pennsylvania State University Combustion instability detection
Authors: Devesh K. Jha, Nurali Virani and Asok Ray This work has been supported by U.S. Air Force Office of Scientific Research (AFOSR) under Grant No. FA9550-15-1-0400 (Dynamic Data-driven Application Systems; PM: Dr. Frederica Darema)
Learning Information Representations for Time-Series Data
Combustion Instability
System health monitoring
Flow regime in nuclear plant heat exchangers
Challenges?
Motivating Problems:
• Mathematical characterization and useful representation of data (Data to knowledge) • Inference of accurate model structure and efficient estimation of model parameters (Modeling and analysis of hyper-parameters)
• Rotorcraft stability monitoring, structural health monitoring, combustion instability detection, target classification with seismic sensors, two-phase flow regime classification using ultrasonic, and battery state estimation
How can we create compact representation of time-series data (using Markov models)?
Markov models for information representation from time-series data How to discretize data? 1. Alphabet size? 2. Location of partitions? 1. Uniform 2. Max-Entropy
…….𝑠1 𝑠2 𝑠1 𝑠1 𝑠2 𝑠1 𝑠2 𝑠2 𝑠1 𝑠1 𝑠1 …..
Markov Modeling? 1. Order Estimation? 2. Parameter Estimation?
Symbolic Time-series Analysis
Applications Anomaly/Change/Fault Detection
Prognostics and Health Monitoring
Activity Recognition
Sensor Fusion for Event Detection
Definition of Probabilistic Finite State Automaton (PFSA) as a generative model
Each state is a collection of symbols
Collection of memory words once memory (depth) is estimated, e.g., Q={11,12,21,22} with A={1,2} and D=2
Deterministic algebraic structure
Sufficient Statistic for the Markov model Advantages 1. Simple to infer model structure 2. Simple to estimate model parameters like symbol emission probabilities (as compared to using Dynamic programming in Hidden Markov Models)
Order Estimation for Markov Modeling Definition: Let be the observed symbol sequences where each . Then, the order (or depth) of the process generating is defined as the length such that :
Most techniques follow a wrapper like search approach Build Model Find Performance Select the best one. Log-Likelihood based Order Estimation
Signal reconstruction-based : Make models in symbolic domain Make predictions in symbolic space Generate predictions in the continuous domain Pick the best one Prediction by a model Euclidean metric
Order Estimation for Markov Modeling Definition: Let be the observed symbol sequences where each . Then, the order (or depth) of the process generating is defined as the length such that :
Most techniques follow a wrapper like search approach Build Model Find Performance Select the best one. Log-Likelihood based Order Estimation
Signal reconstruction-based : Make models in symbolic domain Make predictions in symbolic space Generate predictions in the continuous domain Pick the best one Prediction by a model Euclidean metric Entropy rate-based
Order Estimation for Markov Modeling*
Approximate estimate
*D.K. Jha, A. Srivastav, K. Mukherjee and A. Ray: “Depth Estimation in Markov models of Time-Series Data via Spectral Analysis”, in American Control Conference, 2015 ** D.K. Jha, A. Srivastav and A. Ray: “Depth Estimation in Markov models of Time-Series Data”, in preparation
Reduced Order Markov Modeling: State Merging with deterministic structure?
The deterministic algebraic structure of the finite-state probabilistic graph constraints the state merging process Bigger state space requires more data for convergence, slower performance
2, 0 (?) 2
22
02
1
0
21 1
01
1 (?)
2
01 0 (?)
20
00
0
2, 0
1 (?)
21
02; 20 2 (?)
22
01; 21
1
02; 20; 00; 22
00
Nondeterminism (?) Is that a problem?
The targets must be merged together Targets may not be statistically similar; Leads to statistical modeling error?
Reduced Order Markov Modeling: State Merging with Nondeterministic Structure Remove the algebraic constraints and allow non-determinism Merge states based on statistical distance 𝑠1 𝑠2 …
𝑠𝑁
Set of symbols
Dynamic Bayesian Network
… 1. Non-deterministic probabilistic graph 2. Stopping rule (e.g., allowed model distortion or number of states in the final model) 3. Parameter estimation using dynamic factored graphical model 1
1
1
(for example) Nondeterministic Structure
A Primer to Combustion Instability Dynamics Thermo-acoustic Feedback Cycle Low NOx emission regulation Low equivalence ratio combustion Prone to instabilities VERY FAST REACTIVE DYNAMICS (~10−3 s)
Heat Release Rate Fluctuation (Combustion), q
COMPLEX VELOCITY, THERMAL & ACOUSTIC COUPLING
Velocity Fluctuation (Flow Dynamics), u
Pressure Fluctuation (Combustor acoustics), p
𝐷𝐮 1 + 𝛻𝑝 = 𝟎 𝐷𝑡 𝜌0 OVERALL MECHANISM OF THERMO-ACOUSTIC INSTABILITY IN COMBUSTOR
NONLINEAR COUPLED DYNAMICS
Schematic of the Combustor Apparatus at Penn State PENN STATE CENTER FOR COMBUSTION, POWER AND PROPULSION Prof. Dominic Santavicca and coworkers Test Apparatus for Methane Gas Combustion
STABLE
Approximate Empirical Density
UNSTABLE
Combustion Instability Dynamics
STABLE COMBUSTION
UNSTABLE COMBUSTION
Pressure Data during Lean-Premixed Combustion: Modeling
Coarse graining of data results in lots of self-loop Down-sampling is required
Find the statistics for the model with D= Different Model structures and parameters in different operating regimes Reflects changes in the temporal model of data
Reduced Order Markov Modeling: Comparison and Further Insights Statistically similar states suggest Symbolic Noise
Stable Combustion Pressure Data
K-L distance between symbol emissions from different states
States are significantly different Informative Markov chain
Unstable Combustion Pressure Data K-L distance between conditional symbol Chains Information Theoretic Metrics for Comparing Complexity of Markov emission vs. marginal Maximum Cluster Divergence : symbol emission Discrepancy between i.i.d. and Markov Statistics (Information gain by Markov Models)
Anomaly Detection during Combustion: Results
Combustion Instability Detection: Departure from Stability Stable Case
Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 3 states Cluster of the stable behavior
Behavior in Information Space
Detection of departure from stable behavior
Receiver Operating Curves (ROC) for different Metrics and Model Parameters
Unstable Case
Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 3 states Unstable behavior sticks to the edges of the simplex
Combustion Instability Detection: Detection of Unstable Phase Detection of Unstable Behavior Different from detection of departure from stable
Reduced Model with Two States on Simplex Plane Unstable Behavior
Train Gaussian Process (GP) to learn the manifold
Estimate Likelihoods for Stable and Unstable Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 2 states
Stable Behavior
Instability Detection by the GP model Reduced Order Models perform equally good Reduced Order Model have lesser number of parameters to estimate Faster during test
Concluding Remarks Conclusions Compact representation for Markov models of time-series data by state aggregation Spectral analysis provides a computationally efficient technique for memory estimation State-aggregation using agglomerative clustering with symmetric K-L distance Final model is a non-deterministic finite state automata Experimental validation of the approach using pressure time-series from an unstable combustion process in a swirl-stabilized combustor Comparable performance of reduced-order models observed
Future Research Use of ideas from Information theory like minimum description length (MDL) for model selection to terminate state aggregation Simultaneous search of the associated hyper-parameters for symbolic dynamics-based Markov modeling (partitions and order)
Thank You
Markov Modeling of Time-Series Data via Spectral Analysis Nurali Virani Department of Mechanical Engineering The Pennsylvania State University Combustion instability detection
Authors: Devesh K. Jha, Nurali Virani and Asok Ray This work has been supported by U.S. Air Force Office of Scientific Research (AFOSR) under Grant No. FA9550-15-1-0400 (Dynamic Data-driven Application Systems; PM: Dr. Frederica Darema)
Learning Information Representations for Time-Series Data
Combustion Instability
System health monitoring
Flow regime in nuclear plant heat exchangers
Challenges?
Motivating Problems:
• Mathematical characterization and useful representation of data (Data to knowledge) • Inference of accurate model structure and efficient estimation of model parameters (Modeling and analysis of hyper-parameters)
• Rotorcraft stability monitoring, structural health monitoring, combustion instability detection, target classification with seismic sensors, two-phase flow regime classification using ultrasonic, and battery state estimation
How can we create compact representation of time-series data (using Markov models)?
Markov models for information representation from time-series data How to discretize data? 1. Alphabet size? 2. Location of partitions? 1. Uniform 2. Max-Entropy
…….𝑠1 𝑠2 𝑠1 𝑠1 𝑠2 𝑠1 𝑠2 𝑠2 𝑠1 𝑠1 𝑠1 …..
Markov Modeling? 1. Order Estimation? 2. Parameter Estimation?
Symbolic Time-series Analysis
Applications Anomaly/Change/Fault Detection
Prognostics and Health Monitoring
Activity Recognition
Sensor Fusion for Event Detection
Definition of Probabilistic Finite State Automaton (PFSA) as a generative model
Each state is a collection of symbols
Collection of memory words once memory (depth) is estimated, e.g., Q={11,12,21,22} with A={1,2} and D=2
Deterministic algebraic structure
Sufficient Statistic for the Markov model Advantages 1. Simple to infer model structure 2. Simple to estimate model parameters like symbol emission probabilities (as compared to using Dynamic programming in Hidden Markov Models)
Order Estimation for Markov Modeling Definition: Let be the observed symbol sequences where each . Then, the order (or depth) of the process generating is defined as the length such that :
Most techniques follow a wrapper like search approach Build Model Find Performance Select the best one. Log-Likelihood based Order Estimation
Signal reconstruction-based : Make models in symbolic domain Make predictions in symbolic space Generate predictions in the continuous domain Pick the best one Prediction by a model Euclidean metric
Order Estimation for Markov Modeling Definition: Let be the observed symbol sequences where each . Then, the order (or depth) of the process generating is defined as the length such that :
Most techniques follow a wrapper like search approach Build Model Find Performance Select the best one. Log-Likelihood based Order Estimation
Signal reconstruction-based : Make models in symbolic domain Make predictions in symbolic space Generate predictions in the continuous domain Pick the best one Prediction by a model Euclidean metric Entropy rate-based
Order Estimation for Markov Modeling*
Approximate estimate
*D.K. Jha, A. Srivastav, K. Mukherjee and A. Ray: “Depth Estimation in Markov models of Time-Series Data via Spectral Analysis”, in American Control Conference, 2015 ** D.K. Jha, A. Srivastav and A. Ray: “Depth Estimation in Markov models of Time-Series Data”, in preparation
Reduced Order Markov Modeling: State Merging with deterministic structure?
The deterministic algebraic structure of the finite-state probabilistic graph constraints the state merging process Bigger state space requires more data for convergence, slower performance
2, 0 (?) 2
22
02
1
0
21 1
01
1 (?)
2
01 0 (?)
20
00
0
2, 0
1 (?)
21
02; 20 2 (?)
22
01; 21
1
02; 20; 00; 22
00
Nondeterminism (?) Is that a problem?
The targets must be merged together Targets may not be statistically similar; Leads to statistical modeling error?
Reduced Order Markov Modeling: State Merging with Nondeterministic Structure Remove the algebraic constraints and allow non-determinism Merge states based on statistical distance 𝑠1 𝑠2 …
𝑠𝑁
Set of symbols
Dynamic Bayesian Network
… 1. Non-deterministic probabilistic graph 2. Stopping rule (e.g., allowed model distortion or number of states in the final model) 3. Parameter estimation using dynamic factored graphical model 1
1
1
(for example) Nondeterministic Structure
A Primer to Combustion Instability Dynamics Thermo-acoustic Feedback Cycle Low NOx emission regulation Low equivalence ratio combustion Prone to instabilities VERY FAST REACTIVE DYNAMICS (~10−3 s)
Heat Release Rate Fluctuation (Combustion), q
COMPLEX VELOCITY, THERMAL & ACOUSTIC COUPLING
Velocity Fluctuation (Flow Dynamics), u
Pressure Fluctuation (Combustor acoustics), p
𝐷𝐮 1 + 𝛻𝑝 = 𝟎 𝐷𝑡 𝜌0 OVERALL MECHANISM OF THERMO-ACOUSTIC INSTABILITY IN COMBUSTOR
NONLINEAR COUPLED DYNAMICS
Schematic of the Combustor Apparatus at Penn State PENN STATE CENTER FOR COMBUSTION, POWER AND PROPULSION Prof. Dominic Santavicca and coworkers Test Apparatus for Methane Gas Combustion
STABLE
Approximate Empirical Density
UNSTABLE
Combustion Instability Dynamics
STABLE COMBUSTION
UNSTABLE COMBUSTION
Pressure Data during Lean-Premixed Combustion: Modeling
Coarse graining of data results in lots of self-loop Down-sampling is required
Find the statistics for the model with D= Different Model structures and parameters in different operating regimes Reflects changes in the temporal model of data
Reduced Order Markov Modeling: Comparison and Further Insights Statistically similar states suggest Symbolic Noise
Stable Combustion Pressure Data
K-L distance between symbol emissions from different states
States are significantly different Informative Markov chain
Unstable Combustion Pressure Data K-L distance between conditional symbol Chains Information Theoretic Metrics for Comparing Complexity of Markov emission vs. marginal Maximum Cluster Divergence : symbol emission Discrepancy between i.i.d. and Markov Statistics (Information gain by Markov Models)
Anomaly Detection during Combustion: Results
Combustion Instability Detection: Departure from Stability Stable Case
Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 3 states Cluster of the stable behavior
Behavior in Information Space
Detection of departure from stable behavior
Receiver Operating Curves (ROC) for different Metrics and Model Parameters
Unstable Case
Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 3 states Unstable behavior sticks to the edges of the simplex
Combustion Instability Detection: Detection of Unstable Phase Detection of Unstable Behavior Different from detection of departure from stable
Reduced Model with Two States on Simplex Plane Unstable Behavior
Train Gaussian Process (GP) to learn the manifold
Estimate Likelihoods for Stable and Unstable Each point is a row of symbol emission matrix (with |A|=3) Reduced Model with 2 states
Stable Behavior
Instability Detection by the GP model Reduced Order Models perform equally good Reduced Order Model have lesser number of parameters to estimate Faster during test
Concluding Remarks Conclusions Compact representation for Markov models of time-series data by state aggregation Spectral analysis provides a computationally efficient technique for memory estimation State-aggregation using agglomerative clustering with symmetric K-L distance Final model is a non-deterministic finite state automata Experimental validation of the approach using pressure time-series from an unstable combustion process in a swirl-stabilized combustor Comparable performance of reduced-order models observed
Future Research Use of ideas from Information theory like minimum description length (MDL) for model selection to terminate state aggregation Simultaneous search of the associated hyper-parameters for symbolic dynamics-based Markov modeling (partitions and order)
Thank You
