Dynamic Data-Driven Avionics Systems: Inferring Failure Modes from Data Streams Shigeru Imai, Alessandro Galli, and Carlos A. Varela Worldwide Computing Laboratory Rensselaer Polytechnic Institute
1
DDDAS Workshop at ICCS Reykjavik, Iceland, June 2015
Outline Dynamic Data-Driven Avionics Systems
PILOTS
(ProgrammIng Language for SpatiO-Temporal Streaming applications)
Tuninter 1153 Flight Accident
Anomaly Detection Model
Evaluation Conclusion/Future work
2
PILOTS System
PILOTS* embodies Dynamic Data-Driven Avionics Systems PILOTS System Corrected inputs
Aircraft sensors
Avionics Application DDDAS Steering Loop
Corrected outputs
Airplane Pilot/Cont roller (User)
Measured error
Error Detection & Data Correction
Identified failure
(Mathematical function patterns used to identify failure modes)
*: ProgrammIng Language for spatiO-Temporal data Streaming applications
3
PILOTS Programming Language
Highly declarative
First class support for spatiotemporal input data selection Homogenize (often sparse) existing data to application’s queries closest, euclidean, interpolate methods
Error detection and correction Define error conditions as error signatures 4
program Twice; inputs a(t) using closest(t); b(t) using closest(t); outputs o: b - 2*a at every 1 sec; errors e: b - 2*a; signatures “Normal”; S0 :e = 0 S1(K):e = 2*t+K “A failure”; S2(K):e =-2*t+K “B failure”; S3(K):e = K, abs(K) > 20 “Out-of-sync”; correct S1: a = b / 2; S2: b = a * 2; end
Error Detection Algorithm in PILOTS Measured error
Error signatures
e(t) t
ω: window size
No error : S0
S1
SM
(1) Compute L1 distance between e(t) and each signature
δ =
(2) Convert the distances to a mode likelihood vector
L = 0.60>
(3) Choose the best matching signature 5
…
-1: unknown error
Each element:
If two or more satisfy significance s threshold τ
Air France AF447
6
7/8/2015
Outline Dynamic Data-Driven Avionics Systems
PILOTS
(ProgrammIng Language for SpatiO-Temporal Streaming applications)
Tuninter 1153 Flight Accident
Anomaly Detection Model
Evaluation Conclusion/Future work
7
Tuninter 1153 Flight Accident
Flight from Bari, Italy to Djerba, Tunisia on August 6th, 2005 ATR-72 ditched into the Mediterranean sea
16 of 39 people on board died Bari, Italy
Actual route
x
Planned route
Palermo, Italy
Djerba, Tunisia
http://www.airdisaster.com/photos/ts-lbb/5.shtml “Final Accident Report for TS-LBB” http://www.ansv.it/cgi-bin/eng/FINAL%20REPORT%20ATR%2072.pdf
8
“Mayday” TV Series on Tuninter 1153 https://youtu.be/aCrZwctnNWo?t=1904
Initial Cause of the Accident Incorrect fuel quantity indicator (FQI) installment
FQI for ATR-72 was not working properly (LED failure) Technicians replaced the FQI with one designed for ATR-42
9
FQI showed 2,700 kg of fuel, but fuel actually weighed 550 kg Pilots did not realize data error eventually leading to fuel exhaustion
“Final Accident Report for TS-LBB” http://www.ansv.it/cgi-bin/eng/FINAL%20REPORT%20ATR%2072.pdf
Checking the Validity of Weight
Can we tell if the weight data is invalid from other independently-measured data streams? Relationship between weight and airspeed
10
The lighter, the faster. The heavier, the slower. Is it that simple? -- No
(Actual flight data recovered from flight data recorder (FDR) of the ATR-72 aircraft)
Model Creation Approaches How to relate data streams mathematically? During cruise phase, lift equals weight.
Use a model from aerodynamics theory
1.
Lift:
ρ (air density) is not available from flight data recorder (FDR)
Learn a model from the ATR-72 flight crew operating manual
2. 11
All the required parameters are available from FDR Applicable to other ATR-72 flights
Overview of Our Approach Tuninter 1153
Generally applicable to ATR-72 aircrafts
1st Cruise Phase Data
Tuninter 1153
Tuned to the Tuninter 1153 flight
ATR-72
Evaluation Results
Crew Operating Manual
Model Creation
12
2nd Cruise Phase Data
Model Tuning
Model Evaluation
ATR-72 Flight Crew Operating Manual
Performance tables for take-off, climb, cruise, descent, and landing Focus on cruise this time Weight x1000 [kg]
Altitude x100 [feet])
13
Temp. diff. to ISA [℃]
Airspeed [knot]
Model Creation
Approximate airspeed by 2nd-order polynomial regression
Weight (w), altitude (h), delta ISA (tΔ), airspeed (va)
Correlation = 0.993
14
Model Tuning
Obtained equation not accurate enough Calibrate the equation with the 1st cruise phase using true weight Evaluate with the 2nd cruise phase
15
Underweight Condition
Underweight condition
16
Estimated airspeed ( ) is slower than the actual airspeed ( ) Assuming both va & h are correct, the monitored weight (w) must be too heavy (thus, aircraft is underweight)
True Weight Estimation
Assume no error:
Solve for w
17
PILOTS Program program WeightCheck; /* v_a : airspeed , w: weight , h: altitude */ inputs v_a , w, h(t) using closest (t); outputs corrected_w : w at every 10 sec; errors e: v_a - (6.4869E+01 + 1.4316E-02 * w + 6.6730E-03 * h + (-3.7716E-07) * w * h + (-2.4208E-07) * w * w + (-1.1730E-07) * h * h) + 2.59; signatures 4.69 corresponds to 10% S0(K): e = K, -2 < K, K < 2 "Normal"; discrepancy in weight S1(K): e = K, 4.69 < K "Underweight"; correct S1: w = 3.34523E-12 * (sqrt(1.09278E+22 * h * h + (-1.65342E+27) * h + (-3.69137E+29) * v_a + 1.01119E+32) – 2.32868E+11 * h + 8.83906E+15); end
18
Outline
Dynamic Data-Driven Avionics Systems
PILOTS
(ProgrammIng Language for SpatiO-Temporal Streaming applications)
Tuninter 1153 Flight Accident
Anomaly Detection Model
Evaluation Conclusion/Future work
19
Evaluation
Extract actual flight data from the accident report
Run WeightCheck PILOTS program for 2000~3500 sec of the flight data
Including two cruise phases
Evaluate how accurately it detects/corrects errors in the weight
20
Evaluation period
Results Big discrepancy (≈2000kg) between corrected and real weights during the 2nd cruise phase
Underweight condition is detected 100% for the 2nd cruise phase
21
Outline
Dynamic Data-Driven Avionics Systems
PILOTS
(ProgrammIng Language for SpatiO-Temporal Streaming applications)
Tuninter 1153 Flight Accident
Anomaly Detection Model
Evaluation Conclusions/Future work
22
Conclusions
Weight error detection/correction model for the Tuninter 1153 accident is proposed and evaluated using the PILOTS system Proposed model works well for detecting anomaly conditions in weight (100% accuracy) during cruise phases, but corrected weight is not accurate enough
23
Complex Dependencies Between Data Streams Air France 447 Model vg : ground speed vw : wind speed va : airspeed
w
vg
fq
va vw
h T cf (angle of attack, flaps, landing gear, pitch, roll, yaw)
24
pw Tuninter 1153 Model
fq : fuel quantity w : aircraft weight h : altitude T : temperature pw : engine power cf : aircraft configuration
(Short-term) Future Work Model improvement for the Tuninter accident
Revisit aerodynamics theory
Assuming cruise flight
25
Known constants From data From linear regression
: : : Coefficient of Lift (CL)
http://upload.wikimedia.org/wikipedia/commons/thumb/d /d1/Lift_curve.svg/300px-Lift_curve.svg.png
(Mid-term) Future Work
Expand PILOTS language into a DDDAS Model Learning Toolkit to include:
26
Montecarlo simulation to learn model parameters from data. Kalman filters to reduce the impact of noise in data and enable more robust models. Probabilistic (Bayesian) approach to continuously tune model to data.
(Long-term) Future Work Updated weather
Information from other planes
Cloud-based Offline Data Analytics (to be developed)
3D terrain data
Internet
Safer Flight Assistant System PILOTS*1 System Avionics Application Aircraft sensors
Corrected inputs
Measured error
External data inputs Corrected outputs
Left engine is damaged …
Stochastic & Logic-based Flight Scheduler Failure & (to be developed) Recommended
Airplane pilots
actions
Error Detection & Data Correction (Mathematical function patterns used to identify failure modes)
Identified failure
*1: ProgrammIng Language for spatiO-Temporal data Streaming applications
We should land at airport X immediately!
Fundamental Developments for Safer Flight Assistant System
Quantitative spatial and temporal logic as a formalism:
To enable reasoning about data streams that associate values to specific points or intervals of space and time.
To enable geometric reasoning capabilities, in particular, trigonometric formulae to calculate with aircraft speeds, headings, range, and endurance.
Extensions to logic programming to support stochastic reasoning
Ground speed and crosswind as functions of airspeed, wind, and runway heading
v α
Speed (horizontal)
a
Aircraft
w,x
Wind, crosswind
Direction
r
Runway
Dynamic Data-Driven Flight Plan Adaptation Examples
If…
then…
Language extensions to standard Horn clause-based knowledge bases to incorporate probabilities.
New pilot report: icing en route
New route
Special language support for spatial and temporal data streams.
New winds aloft
New altitude
New surface winds at destination
New airport
Imminent engine failure
Nearest airport
28
Incremental reasoning algorithms to dynamically recompute logical queries efficiently as new data gets injected into the application.
Cloud-based Offline Data Analytics
Scalable correlation analysis from hundreds of independently-measured sensor data streams Automating anomaly detection/correction model creation process
d1 d2
…
d1
… … dN
… Aircraft sensor data streams 29
Cloud Storage
?
d3
? ?
?
…
d2
d1
0.8
?
dN
Virtual Machines Cost-Efficient High-Performance Data Analytics
0.9
d3
…
d1 ≈ c∙d2
d2 0.6
dN
Open Source Software
Download PILOTS 0.2.4 at: http://wcl.cs.rpi.edu/pilots Thanks! Questions? Partial support from: Air Force Office of Scientific Research DDDAS Program Dr. Frederica Darema (AFOSR Grant No. FA9550-15-1-0214) & Yamada Corporation Fellowship
30
MIT Press, June 2013
Extra Slides
31
Weight Airspeed relationship for Multiple Flight Levels
32
Analysis of Discrepancy
Proposed weight model is sensitive to airspeed change 3140 sec
3280 sec
+2.46knots
-1402kg
(16070.8, 181.94)
(14668.8, 184.4)
33
Analysis of discrepancy (cont.)
Actual weight: insensitive to airspeed change Corrected weight: sensitive to airspeed change
34