Varela DDDAS Accomplishments 2017
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute
•
Dynamic Data-Driven Avionics Systems: Sensor data streaming analytics, high-level programming models and software for DDDAS, failure modeling, situational awareness, decision support during flight emergencies.
– •
Key Focus of Scientific Research
– • • •
AF Relevance: Autonomous Systems, Autonomous Reasoning and Learning, Autonomous Mission Planning, Decision Support Tools. Dynamic data-driven fault modeling, DDDAS software for spatio-temporal stream analytics, declarative programming, distributed computing and reasoning with probabilistic uncertain data and models.
Colleagues: Erik Blasch, Randolph Franklin, Stacy Patterson. Graduate students: Shigeru Imai, Saswata Paul, Wennan Zhu. Undergraduate students: Sida Chen, Alessandro Galli, David Glowny, Frederick Hole, Frederick Lee, Liyu Pan, Pratik Patel, Alexandra Zytek.
1
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute Updated weather
Previous flights data
Information from other planes
Cloud
Expert-Level Flight Assistant Offline Training
Prediction response
Avionics Application
Aircraft sensors Corrected inputs
(Mathematical function patterns used to identify failure modes)
External real-time data inputs
PILOTS* Learning Engine System
Prediction request
3D terrain data
Measured error
Failure Detection & Data Correction
Left engine is damaged …
Corrected outputs
Probabilistic Logic Flight Assistant (in development) Identified failure
*: ProgrammIng Language for spatiO-Temporal data Streaming applications
Airplane pilots Failure & Recommended actions We should land at airport X immediately!
2
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute
Using a feedback loop, Dynamic Data-Driven Avionics Systems continuously analyze spatio-temporal data streams coming from airplane sensors, identifying potential failure modes, correcting erroneous data when possible, and suggesting best flight plans to pilots. The resulting capability is a new layer of expert advice and logical redundancy in addition to existing physical redundancy for safer flight systems.
1.Failure modeling advances: a.
Error signatures
• b.
Mathematical function patterns with constraints arising on specific (failure-induced) data stream errors/anomalies. Mode likelihood vectors
• c.
Stochastic selection of DDDAS system operation mode based on well-behaved vectors of error signatures. Data-driven learning
• •
Machine learning (Montecarlo, regression) to infer failure models (error functions and signatures) from data. Bayesian learning to update probability distribution functions in failure models with new evidence (data).
2.New DDDAS software: PILOTS programming language, learning toolkit, and run-time system a.
Enables declarative (high-level) definition of DDDAS data streaming application models (input-output relationships between data streams), error signatures, and error correction functions.
b.
PILOTS software detects specific (e.g., failure-induced) data errors based on their signatures and autonomously corrects data before processing it according to the application model.
c. d.
Modular scalable software development to seamlessly incorporate different data-driven machine learning techniques. Dynamic data-driven trajectory generation based on Dubins paths for loss of thrust scenarios.
3.We have applied the developed DDDAS avionics concepts/software to the following scenarios: a.
Air France AF447 accident in June 2009: The airspeed sensor failure of the AF447 flight is successfully detected and corrected after 5 seconds from the beginning of the failure. Overall error mode detection accuracy reaches 96.31%.
b.
Tuninter 1153 accident in August 2005: The underweight condition due to the installation of an incorrect fuel sensor is successfully detected with 100% accuracy during the cruise phase of flight.
c.
US Airways 1549 incident in January 2009: Bird strike after takeoff requiring real-time trajectory generation assistance. We can successfully generate trajectories up to 36 seconds after birds strike. Sequential computation time: 120ms per trajectory.
d.
Elastic scalable (decentralized) air traffic optimization using data streams and cloud computing. 3
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute Publications 1)#1, #2, #3
Shigeru Imai, Erik Blasch, Alessandro Galli, Wennan Zhu, Frederic Lee, and Carlos A. Varela. Airplane Flight Safety using Error-Tolerant Data Stream Processing. IEEE Aerospace & Electronic Systems Magazine (AESM), 32(4), pp. 4-17, April 2017.
2)#2d, #3c
Saswata Paul, Frederick Hole, Alexandra Zytek, and Carlos A. Varela. Flight Trajectory Planning for Fixed-Wing Aircraft in Loss of Thrust Emergencies. Dynamic Data-Driven Application Systems (DDDAS 2017), Cambridge, MA, August 2017.
3)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Maximum Sustainable Throughput Prediction for Data Stream over Public Clouds. In 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2017), Madrid, Spain, May 2017.
4)#2c
Carlos Gomez, Harold Castro, and Carlos A. Varela. Global Snapshot of a Distributed System running on Virtual Machines. In International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD 2017), Campinas, Brasil, October 2017, to appear.
Processing
5)#1c, #2c
, #3b
Sida Chen, Shigeru Imai, Wennan Zhu, and Carlos A. Varela. Towards Learning Spatio-Temporal Data Stream Relationships for Failure Detection in Avionics. Dynamic Data-Driven Application Systems (DDDAS 2016), Hartford, CT, August 2016.
6)#3d
Shigeru Imai, Pratik Patel, and Carlos A. Varela. Developing Elastic Software for the Cloud. In Encyclopedia on Cloud Computing. Wiley, 2016.
7)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Cost-Efficient Elastic Stream Processing Using Application- Agnostic Performance Prediction. In 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2016), Cartagena, Colombia, May 2016. Best doctoral symposium paper award.
8)#2c, #3d
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Elastic Virtual Machine Scheduling for Continuous Air Traffic Optimization. In 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2016), Cartagena, Colombia, May 2016.
9)#3d
Stacy Patterson and Carlos A. Varela. Steering Complex Systems using a Dynamic Data-Driven Modeling Approach. Streaming Technology Requirements, Application and Middleware (STREAM2016), Washington, DC, March 2016.
10)#2c
Travis Desell and Carlos A. Varela. A Performance and Scalability Analysis of Actor Message Passing and Migration in SALSA Lite. In Agere Workshop at ACM SPLASH 2015 Conference, October 2015.
11)#2c
Matthew Hancock and Carlos A. Varela. Augmenting Performance For Distributed Cloud Storage. In 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015), Shenzhen, China, May 2015.
12)#1, #2, #3
Shigeru Imai, Alessandro Galli, and Carlos A. Varela. Dynamic Data-Driven Avionics Systems: Inferring Failure Modes from Data Streams. In Dynamic Data-Driven Application Systems (DDDAS 2015), Reykjavik, Iceland, June 2015.
13)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Cost-Efficient High-Performance Internet-Scale Data Analytics over MultiCloud Environments. In 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015), Shenzhen, China, May 2015.
14)#2c
Matthew Hancock. Middleware Framework for Distributed Cloud Storage. Master's thesis, Rensselaer Polytechnic Institute, May 2015.
15)#1, #2, #3
Carlos A Varela. Dynamic Data Driven Avionics Systems. Streaming Technology Requirements, Application and Middleware (STREAM2015), Indianapolis, IN, October 2015. 4
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute
• Coordination/Synergy – Visited AFRL, Rome. August 2017. Participants: • Erik Blasch • Albert Frantz • Joseph Raquepas • Stanley Wenndt • Other PIs: Matt Hoffman, Anthony Vodacek, RIT. – Discussed potential research collaboration: use of hyperspectral data to detect safe emergency landing areas.
– Discussed potential SBIR, STTR funding, data/software coordination, summer collaborations, AFRL/DOD transition efforts. 5
•
Dynamic Data-Driven Avionics Systems: Sensor data streaming analytics, high-level programming models and software for DDDAS, failure modeling, situational awareness, decision support during flight emergencies.
– •
Key Focus of Scientific Research
– • • •
AF Relevance: Autonomous Systems, Autonomous Reasoning and Learning, Autonomous Mission Planning, Decision Support Tools. Dynamic data-driven fault modeling, DDDAS software for spatio-temporal stream analytics, declarative programming, distributed computing and reasoning with probabilistic uncertain data and models.
Colleagues: Erik Blasch, Randolph Franklin, Stacy Patterson. Graduate students: Shigeru Imai, Saswata Paul, Wennan Zhu. Undergraduate students: Sida Chen, Alessandro Galli, David Glowny, Frederick Hole, Frederick Lee, Liyu Pan, Pratik Patel, Alexandra Zytek.
1
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute Updated weather
Previous flights data
Information from other planes
Cloud
Expert-Level Flight Assistant Offline Training
Prediction response
Avionics Application
Aircraft sensors Corrected inputs
(Mathematical function patterns used to identify failure modes)
External real-time data inputs
PILOTS* Learning Engine System
Prediction request
3D terrain data
Measured error
Failure Detection & Data Correction
Left engine is damaged …
Corrected outputs
Probabilistic Logic Flight Assistant (in development) Identified failure
*: ProgrammIng Language for spatiO-Temporal data Streaming applications
Airplane pilots Failure & Recommended actions We should land at airport X immediately!
2
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute
Using a feedback loop, Dynamic Data-Driven Avionics Systems continuously analyze spatio-temporal data streams coming from airplane sensors, identifying potential failure modes, correcting erroneous data when possible, and suggesting best flight plans to pilots. The resulting capability is a new layer of expert advice and logical redundancy in addition to existing physical redundancy for safer flight systems.
1.Failure modeling advances: a.
Error signatures
• b.
Mathematical function patterns with constraints arising on specific (failure-induced) data stream errors/anomalies. Mode likelihood vectors
• c.
Stochastic selection of DDDAS system operation mode based on well-behaved vectors of error signatures. Data-driven learning
• •
Machine learning (Montecarlo, regression) to infer failure models (error functions and signatures) from data. Bayesian learning to update probability distribution functions in failure models with new evidence (data).
2.New DDDAS software: PILOTS programming language, learning toolkit, and run-time system a.
Enables declarative (high-level) definition of DDDAS data streaming application models (input-output relationships between data streams), error signatures, and error correction functions.
b.
PILOTS software detects specific (e.g., failure-induced) data errors based on their signatures and autonomously corrects data before processing it according to the application model.
c. d.
Modular scalable software development to seamlessly incorporate different data-driven machine learning techniques. Dynamic data-driven trajectory generation based on Dubins paths for loss of thrust scenarios.
3.We have applied the developed DDDAS avionics concepts/software to the following scenarios: a.
Air France AF447 accident in June 2009: The airspeed sensor failure of the AF447 flight is successfully detected and corrected after 5 seconds from the beginning of the failure. Overall error mode detection accuracy reaches 96.31%.
b.
Tuninter 1153 accident in August 2005: The underweight condition due to the installation of an incorrect fuel sensor is successfully detected with 100% accuracy during the cruise phase of flight.
c.
US Airways 1549 incident in January 2009: Bird strike after takeoff requiring real-time trajectory generation assistance. We can successfully generate trajectories up to 36 seconds after birds strike. Sequential computation time: 120ms per trajectory.
d.
Elastic scalable (decentralized) air traffic optimization using data streams and cloud computing. 3
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute Publications 1)#1, #2, #3
Shigeru Imai, Erik Blasch, Alessandro Galli, Wennan Zhu, Frederic Lee, and Carlos A. Varela. Airplane Flight Safety using Error-Tolerant Data Stream Processing. IEEE Aerospace & Electronic Systems Magazine (AESM), 32(4), pp. 4-17, April 2017.
2)#2d, #3c
Saswata Paul, Frederick Hole, Alexandra Zytek, and Carlos A. Varela. Flight Trajectory Planning for Fixed-Wing Aircraft in Loss of Thrust Emergencies. Dynamic Data-Driven Application Systems (DDDAS 2017), Cambridge, MA, August 2017.
3)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Maximum Sustainable Throughput Prediction for Data Stream over Public Clouds. In 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2017), Madrid, Spain, May 2017.
4)#2c
Carlos Gomez, Harold Castro, and Carlos A. Varela. Global Snapshot of a Distributed System running on Virtual Machines. In International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD 2017), Campinas, Brasil, October 2017, to appear.
Processing
5)#1c, #2c
, #3b
Sida Chen, Shigeru Imai, Wennan Zhu, and Carlos A. Varela. Towards Learning Spatio-Temporal Data Stream Relationships for Failure Detection in Avionics. Dynamic Data-Driven Application Systems (DDDAS 2016), Hartford, CT, August 2016.
6)#3d
Shigeru Imai, Pratik Patel, and Carlos A. Varela. Developing Elastic Software for the Cloud. In Encyclopedia on Cloud Computing. Wiley, 2016.
7)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Cost-Efficient Elastic Stream Processing Using Application- Agnostic Performance Prediction. In 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2016), Cartagena, Colombia, May 2016. Best doctoral symposium paper award.
8)#2c, #3d
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Elastic Virtual Machine Scheduling for Continuous Air Traffic Optimization. In 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2016), Cartagena, Colombia, May 2016.
9)#3d
Stacy Patterson and Carlos A. Varela. Steering Complex Systems using a Dynamic Data-Driven Modeling Approach. Streaming Technology Requirements, Application and Middleware (STREAM2016), Washington, DC, March 2016.
10)#2c
Travis Desell and Carlos A. Varela. A Performance and Scalability Analysis of Actor Message Passing and Migration in SALSA Lite. In Agere Workshop at ACM SPLASH 2015 Conference, October 2015.
11)#2c
Matthew Hancock and Carlos A. Varela. Augmenting Performance For Distributed Cloud Storage. In 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015), Shenzhen, China, May 2015.
12)#1, #2, #3
Shigeru Imai, Alessandro Galli, and Carlos A. Varela. Dynamic Data-Driven Avionics Systems: Inferring Failure Modes from Data Streams. In Dynamic Data-Driven Application Systems (DDDAS 2015), Reykjavik, Iceland, June 2015.
13)#2c
Shigeru Imai, Stacy Patterson, and Carlos A. Varela. Cost-Efficient High-Performance Internet-Scale Data Analytics over MultiCloud Environments. In 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2015), Shenzhen, China, May 2015.
14)#2c
Matthew Hancock. Middleware Framework for Distributed Cloud Storage. Master's thesis, Rensselaer Polytechnic Institute, May 2015.
15)#1, #2, #3
Carlos A Varela. Dynamic Data Driven Avionics Systems. Streaming Technology Requirements, Application and Middleware (STREAM2015), Indianapolis, IN, October 2015. 4
Active Data: A New Paradigm for Data Streaming Analytics PI: Carlos Varela, Rensselaer Polytechnic Institute
• Coordination/Synergy – Visited AFRL, Rome. August 2017. Participants: • Erik Blasch • Albert Frantz • Joseph Raquepas • Stanley Wenndt • Other PIs: Matt Hoffman, Anthony Vodacek, RIT. – Discussed potential research collaboration: use of hyperspectral data to detect safe emergency landing areas.
– Discussed potential SBIR, STTR funding, data/software coordination, summer collaborations, AFRL/DOD transition efforts. 5
