Hidden Markov Model-Based Real-Time Transient ... - Semantic Scholar
Hidden Markov Model-Based Real-Time Transient Identifications in Nuclear Power Plants Kee-Choon Kwon,1,* Jin-Hyung Kim,2 Poong-Hyun Seong3 1 Korea Atomic Energy Research Institute, P.O. Box 105, Yuseong, Daejeon 305-600, Korea 2 Department of Computer Science, KAIST, 373-1 Gusung-Dong, Yuseong-Gu, Daejeon 305-701, Korea 3 Department of Nuclear Engineering, KAIST, 373-1 Gusung-Dong, Yuseong-Gu, Daejeon 305-701, Korea
In this article, a transient identification method based on a stochastic approach with the hidden Markov model (HMM) has been suggested and evaluated experimentally for the classification of nine types of transients in nuclear power plants (NPPs). A transient is defined as when a plant proceeds to an abnormal state from a normal state. Identification of the types of transients during an early accident stage in NPPs is crucial for proper action selection. The transient can be identified by its unique time-dependent patterns related to the principal variables. The HMM, a double-stochastic process, can be applied to transient identification that is a spatial and temporal classification problem under a statistical pattern-recognition framework. The trained HMM is created for each transient from a set of training data by the maximum-likelihood estimation method which uses a forward-backward algorithm and the Baum-Welch re-estimation algorithm. The transient identification is determined by calculating which model has the highest probability for given test data using the Viterbi algorithm. Several experimental tests have been performed with normalization methods, clustering algorithms, and a number of states in HMM. There are also a few experimental tests that have been performed, including superimposing random noise, adding systematic error, and adding untrained transients to verify its performance and robustness. The proposed real-time transient identification system has been proven to have many advantages, although there are still some problems that should be solved before applying it to an operating NPP. Further efforts are being made to improve the system performance and robustness in order to demonstrate reliability and accuracy to the required level. © 2002 Wiley Periodicals, Inc.
1.
INTRODUCTION
Transient identification in nuclear power plants means classifying the types of transients by interpreting the major plant variables and operating status of the equipment. A transient is defined as when a plant proceeds to an abnormal state from * Author to whom all correspondence should be addressed; e-mail: [email protected]. INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, VOL. 17, 791–811 (2002) © 2002 Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.002/int.10050
792
KWON, KIM, AND SEONG
a normal state. If the error, either caused by a human or plant component, happens to be followed by a fault, then a transient has occurred. Consequently, an accident is preceded by a transient. The term identification, as applied to an engineering system or process, means the classification of the cause that brought either an undesirable state or failure of the system. The identification can be done at several different levels, for example, component, subsystem, function, or event.1 For the proposed transient identification system, identification is made at the event level to determine which transient has occurred in the NPP. It is necessary to identify the type of transient through continuous monitoring of the NPP as a complex dynamic system during its early stage to provide sufficient information to the human operators, in order to assist in proper operator action selection to prevent a more severe situation or to mitigate the accident consequence.2 Also, this system provides input for computerized accident management, computerized operating procedure, and semiautonomous operation systems. Typical transients in NPPs are associated with unique, time-dependent patterns of major variables and equipment status. This time-dependent pattern can be used to identify the transient; hence, identification can be treated as a pattern classification problem.3 The initial trial to identify plant status using pattern classification was the introduction of the safety parameter display system (SPDS) after the Three Mile Island (TMI) accident. It follows the U.S. Nuclear Regulatory Commission (USNRC) Safety Guide NUREG-0696 philosophy.4 It monitors critical plant parameters and concentrates the information in such a way as to give a systematic view of the safety status of the plant, especially under accident conditions. It also assists the operator in his or her diagnosis and decision-making. An example of SPDS is the spider’s webstyle display: the human operator tries to identify the plant’s status by utilizing the unique patterns of parameters from a spider’s web-style display. Another example of SPDS is the Chernoff face, a graphic technique that maps multidimensional data into facial features. The most common usages of the technique are to display the data in a convenient form, to aid in discovering clusters and outliers, and to show changes over time.5,6 In both of the above cases, the displays are provided by machine, but classification is carried out by a human operator. The human operator instantaneously uses only static spatial information, not temporal information. It is difficult for a human operator to identify a transient when the preceding patterns of some transients are very similar and the patterns change further with time. Recently, attempts have been made to solve transient identification problems using a computer-based system. In this transient identification problem, the classification might involve spatial and temporal patterns. Temporal patterns usually involve ordered sequences of data appearing over time. Spatial patterns mean the unique patterns of each transient, variations of the same transient that might occur under different operating modes or at different break sizes. The transient identification systems for NPPs have been developed using techniques such as artificial neural networks (ANNs) and fuzzy logic. Initially, the simplest feedforward backpropagation neural network (NN) techniques were used. Later, many attempts were made to improve identification performance using ANNs to discriminate similar transients,2 reduce the number of training patterns using lateral feedback NNs,7 incorporate modular hierarchy with dynamic node architecture
HIDDEN MARKOV MODEL-BASED REAL-TIME
793
to reduce the size of the system,8 and use probabilistic NN,3 including the “don’t know” answer.9 Simple feedforward NN applications that do not consider the time concept still cannot adequately solve the temporal variations. Recently, there have been many applications that include a time concept such as NN with an implicit time measure,10 recurrent multi-layer perceptron,11 and spatiotemporal NN.12 There are also numerous applications other than ANN, such as fuzzy logic,13 nearest neighbors modeling optimized by using a genetic algorithm,14 adaptive template matching,15 and observer-based residual generation.16 All of these systems are still considered as prototypes or are under evaluation and have not yet been applied to real, operating NPPs. The ANN and fuzzy logic approach can absorb spatial variations, but cannot provide proper solutions for temporal variations. Thus, it is reasonable to adopt a double-stochastic approach for the classification of the patterns. The hidden Markov model (HMM), a double-stochastic process, enables modeling of not only spatial phenomena, but also time-scale distances. The HMM can be used to solve classification problems associated with time series input data such as speech signals or plant process signals, and can provide appropriate solutions by its modeling and learning capabilities, even though it does not have the exact knowledge to solve the problems. Most of the HMM applications for pattern classification in dynamic processes have a typical architecture to solve spatial-temporal problems, but the target systems are different, as in dynamic obstacle avoidance of mobile robot navigation,17 radar target,18 human action,19 American sign language,20 heart signals,21 sonar signals,22 two-handed actions,23 conditions of an electrical machine,24 deep space network antennae,25 moving light displays,26 environmental noise,27 and human genes in DNA.28 But the HMM has never been applied for transient identifications in NPPs. The goal of this article is to provide a real-time transient identification system for use in NPPs that demonstrates the spatial and temporal modeling and learning capabilities with hidden Markov models. To demonstrate these capabilities, simulated NPP parameters are collected and clustered to build hidden Markov models for transient identification purposes. 2.
HIDDEN MARKOV MODEL FOR TRANSIENT IDENTIFICATION
The problem of transient identification is defined as the recognition of transient types, ω j , given the sequential input patterns, X t , at time t. The input pattern, X t , is mathematically defined as an object described by a sequence of features at time t 29 : X t = (x1 , x2 , . . . , xi , . . . , xd )
(1)
The space of input pattern X t consists of the set of all possible patterns: X t ⊂ d ,
where d is a d-dimensional real vector space.
The k observed data up to time t are defined as t−k : t−k = {X t−k+1 , . . . , X t−1 , X t }
(2)
794
KWON, KIM, AND SEONG
The set of possible transient classes, ω j , at time t forms the space of classes, : = {ω1 , ω2 , . . . , ωc },
where c is the number of classes
(3)
The classes, , are assumed to be mutually exclusive and exhaustive. The recognition task can be considered to be the finding of function f , which maps the space of input patterns, t−k , to the space of classes, : f : t−k →
(4)
A dynamic process often exhibits sequentially changing behavior. If the one short time period is defined as a frame, the probability of the frame transition is different from each transient in NPPs. Therefore, the probability of frame existence and the transition between frames can be statistically modeled. The probability of a transient occurring in an NPP is already given and is called the a priori probability. When a transient has occurred in NPPs, the type of transient can be determined only by selecting the type of transient, ω j , with the highest a priori probability, P(ω j ). This decision is obviously unreasonable. It is more reasonable to determine the type of transient after observing the trend of time series major variables, namely, to get the conditional probability, P(ω j | t−k ). This conditional probability is called the a posteriori probability. Decision-making based on the a posteriori probability is more reliable, because it employs both a priori knowledge together with observed time-series data.30 Classification of the unknown pattern, X t , corresponds to finding the optimal model, ω, ˆ that maximizes the conditional probability, P(ω j | t−k ), the probability that the system is in class ω j at time t given that t−k was observed at time t. We can apply Bayes’ rule to calculate the a posteriori probability, P(ωˆ | t−k ) = max ω
P(t−k | ω j )P(ω j ) P(t−k )
(5)
where P(t−k ) =
c
P(t−k | ω j )P(ω j )
(6)
j=1
The conditional probability, observing t−k given that the system is in class ω j at time t, P(t−k | ω j ), comes from comparing the shapes of the transient models with input observations, while the a priori probability, P(ω j ), comes from the transient probability that represents how often the transient appears in the NPP. Because ˆ we get: P(t−k ) is independent of ω, ˆ ω) ˆ P(ωˆ | t−k ) ∝ P(t−k | ω)P( = max[P(t−k | ω j )P(ω j )] ω
(7)
In fact, the a priori probability, P(ω j ), can be calculated in NPPs, and should satisfy the following condition: c j=1
P(ω j ) = 1
(8)
HIDDEN MARKOV MODEL-BASED REAL-TIME
795
But the transient identification system does not cover all of the transients occurring in NPPs, and consequently cannot satisfy Equation 8. Therefore, the present observed data controls the decision. P(t−k | ω j ) is called the likelihood of ωˆ with respect to the set of samples. In real implementation, the a priori probability, P(ω j ), can be assumed that the occurring probabilities of all transients are equal.9 The maximum likelihood estimate of ωˆ is, by definition, that value of ωˆ that maximizes P(t−k | ω j ). The HMM can successfully treat a transient identification problem under a probabilistic or statistical framework.30 The HMM is nothing more than a specific type of parametric probability distribution model. Therefore, all the statistical pattern recognition principles that have been presented above can be applied to HMMs.31 In this identification problem, the HMM is used to estimate the conditional probability, P(t−k | ω j ). By using HMM, the pattern variability in parameter space and time can be modeled effectively. The HMM uses a Markov chain to model changing statistical characteristics that exist in the actual observations of dynamic process signals. The Markov process is therefore a double-stochastic process in which there is an unobservable Markov chain defined by a state transition matrix, and where each state of the Markov chain is associated with a discrete output probability distribution. The double-stochastic processes enable modeling of not only spatial phenomena, but also time-scale distances. Using HMMs in a classification problem with Bayes’ rule and maximum likelihood training requires two things: the evaluation of P(t−k | ω j ) for the implementation of Bayes’ rule, and the maximization of the likelihood for the training of the classifiers. Fortunately, there exist computationally efficient procedures for these two tasks. HMM parameters are estimated from the Baum-Welch algorithm and guarantee a finite improvement on each iteration in the sense of maximization of likelihood. An HMM is trained for each transient from a set of training data and an iterative maximum likelihood estimation of model parameters from observed timeseries data. Incoming observations are classified by calculating which model has the highest probability of producing that observation. An HMM can be represented by the compact notation λ = (A, B, ). Specification of an HMM involves choosing the number of states, N , the number of discrete symbols, L, and the specification of three probability densities with matrix form, A, B, and . The more detailed definition of HMM is described in Ref. 32. Training refers to the characteristics of input patterns to be modeled by the parameter λ = (A, B, ). In this article, the maximum likelihood estimation is used for training. This maximum likelihood estimation of the parameters of a hidden Markov process requires as many observations as possible in a practical implementation. However, in real applications, only finite training data are available, and in some parameters are inadequately trained. Recognition or classification means finding the best path in each trained model and selecting the one that maximizes the path probability for a given input observation. Therefore, given the observation O = O1 , O2 , . . . , OT and the model λi = (Ai , Bi , i ), where i = 1, 2, . . . , C, the classified model λ∗ satisfies the following equation: λ∗ = max P(O | λi ), i
where 1 ≤ i ≤ C
(9)
796
KWON, KIM, AND SEONG
3.
TRANSIENT IDENTIFICATION SYSTEM 3.1.
Pre-processing
Training and test data from the test simulator should be converted to a proper codebook that is input data for the HMM identifier. Pre-processing means to make a codebook, and this method is divided into two techniques: normalization and vector quantization. The input symptom vector to be used for vector quantization has the same value as displayed in a real plant instrument, because it is directly received from the test simulator or the plant computer in NPPs. The range of input data values is significantly different. Therefore, it should be normalized before using the input data of the clustering algorithms. Normalization is one of several transformation techniques. It has the effect of reducing the parameters to a common range. If the distribution of the variable’s values are of normal distribution, it is reasonable to be normalized by maximum value. But as shown in Figure 1, most of the values of the specified variable are distributed in the upper region in the case of normalization by maximum value. It is undesirable to use the input data of the clustering algorithms. It is more reasonable to normalize between its minimum and maximum values, as in the following equation33 : Normalized value (1) = (Value − Min.)/(Max. − Min.)
(10)
In a similar way, mean-centered normalization can be using the following equation: Normalized value (2) = ((Value − Mean)/(0.5 × Range of values) + 1.0) × 0.5 (11) In the mean-centered normalization method, data between the mean and minimum or maximum is linearly distributed. Alternatively, we can define a nonlinear data distribution between the mean and the minimum or maximum range considering the standard deviation using the following equation: Normalized value (3) = ((Value − Mean)/(Cσ ) + 1.0) × 0.5
(12)
C is the constant from the range of values and standard deviation. The results of normalization of the above four methods regarding a specific variable, pressurizer pressure, are shown in Figure 1. In Figure 1, Mean-C1 data are from Equation 11 and Mean-C2 data are from Equation 12. Features can be represented by continuous, discrete, or discrete-binary variables. It is expected that the feature vector contains most of the classification information available from the object.31 Feature extraction is an important task for classification or recognition, and is often necessary as a pre-processing stage for data. In feature extraction, data can be transformed from high-dimensional pattern space to low-dimensional feature space. Vector quantization (VQ), the process of approximating a block of continuous amplitude signals by a discrete signal, is one method of feature extraction. The idea is to quantize each continuous vector to one of a relatively small number of template vectors, which together comprise what is called a codebook. The sequence of codebook indices obtained in this way forms the
797
HIDDEN MARKOV MODEL-BASED REAL-TIME 3,500 max-0
3,000
mean-C1 mean-C2
Distribution
2,500 2,000 1,500 1,000 500 0 0.0
0.2
0.4 0.6 Normalized value
0.8
1.0
Figure 1. Comparison of normalization methods.
desired sequence of discrete symbols. In this article, two VQ methods are introduced to compare the identification capability. To minimize iteratively the average distortion measure, the most widely used method is the K-means algorithm. In the K-means algorithm, the basic idea is to divide the set of training vectors into L clusters, Ci , {1 ≤ i ≤ L}, in such a way that two necessary conditions for optimality are satisfied. The first condition is that the optimal quantization is realized by using a nearest neighbor selection rule. The second condition is that each codeword is chosen to minimize the average distortion.30 The feature mapping algorithm is supposed to convert patterns of arbitrary dimensionality into the response of a one-dimensional array of neurons. Suppose that an input pattern has N features and is represented by a vector x in an n-dimensional pattern space. The network maps the input patterns to an output space. The output space in this case is assumed to be a one-dimensional array of output nodes, which possess a certain topological orderliness. The question is how to train a network so that the ordered relationship can be preserved. Kohonen proposed allowing the output nodes to interact laterally, leading to the self-organizing map (SOM), also known as the Kohonen network.34 3.2.
Real-Time Test Environment
It is more realistic to receive training and test data directly from the operating NPPs. But severe transients or accidents seldom occur in real NPPs, and it is almost impossible to create transient conditions only for experimental purposes. Therefore, it was necessary to use a simulator for operator training, or simulation code ready for safety analysis, to implement and test the transient identification system. In this
798
KWON, KIM, AND SEONG
Modeling program
Real-time executive
Static program
Dynamic program
Shared memory
Print-out selected variables
RUN/FREEZE/ONE_STEP
Get variable value
Command interpreter
Store global variables
Change variable value
Get global variables
Display variables value
Malfunction insert Supervisory program
Figure 2. Software structure of test simulator.
implementation, the test simulator was modified from the compact nuclear simulator, installed at the Korea Atomic Energy Research Institute Nuclear Training Center for training non-operator personnel, for testing the transient identification system. The test simulator is divided into two major parts: a mathematical modeling program, which executes the plant dynamic modeling program in real-time, and a supervisory program that manages user instructions. The simplified software structure of the test simulator is depicted in Figure 2. The NPP modeled in the test simulator is a three-loop, 993 MWe Westinghouse pressurized water reactor, mostly used in the Kori Units 3 and 4 in Korea. The mathematical modeling programs consist of static and dynamic parts. The initial state, a 100 percent full-power condition is set up in the static calculation, which is performed once before the start of the dynamic calculation. The dynamic calculation is performed every 0.2 second to represent a real-time mathematical modeling simulation. The test simulator provides the function to activate 79 predefined malfunctions. This function realizes the transient or accident condition to get training data and to test the transient identification system.35 3.3.
Data Collection
The transients or accidents in NPPs can be categorized by the required operator response, the required plant protection system actuation, expected frequency of occurrence, or postulated initiating event. The nine typical transients are selected
HIDDEN MARKOV MODEL-BASED REAL-TIME
799
among different postulated transients that might occur in NPPs considering the simulation capability of the test simulator, because the purpose of this article is to demonstrate the capability of hidden Markov models that apply to transient identification problems. The detailed descriptions of the target transients are as follows: (1) ATWS (anticipated transient without scram) The scram signal is lost, caused by a reactor protection logic circuit failure at any power level. The control rods are not moving down on the scram signal. (2) FWLB (feedwater line break inside containment) The feedwater line is broken inside containment. (3) LOCA (small loss-of-coolant accident) A leakage of primary coolant into containment. A break is assumed in the primary loop as a partial rupture of the pipe itself or as a rupture of the smaller pipes connected into the primary loop. (4) LSLC (loss of steam generator-level controller signal) Steam generator level control signal is lost caused by steam generator level controller failure. (5) MSIV (main steam isolation valve closure) The main steam isolation valve is closed, caused by a driving mechanism failure. (6) MSLI (main steam line break inside containment) The whole main steam line pipe or a part of it is broken. Because the secondary isolation valves are located in the turbine hall, the possible combination is nonisolable leak inside containment. (7) MSLO (main steam line break outside containment) The same as MSLI, when the possible combination is nonisolable leak outside containment. (8) PORV (power-operated relief valve stuck open) The pressurizer power-operated relief valve is stuck open, caused by control mechanism failures. (9) SGTR (steam generator tube rupture) A rupture of the steam generator tubes. As a consequence of thermal stresses on the tube material, one or several steam generator tubes are broken, opening a flow connection between the primary and secondary loops.
The input symptom vector is a collection of the principal variables and the status of major equipment from the transient simulation in the test simulator. The major variables and equipment status used to identify the nine different types of transients and one normal state are summarized in Table I. The selection of variables is dependent on the knowledge of an experienced reactor operator and accident analysis from the safety analysis report (SAR). There are two types of data, training data and test data. The training data are needed to train the clustering algorithm and HMM identifier. To test the classification capability of the HMM identifier, test data are also needed. The training and test data are collected from the test simulator. It is needed to get more widely spread training data per transient to design a reasonable classifier. In this experiment, we considered different operating modes and different break sizes in each transient. The HMM might absorb the variations from the different operating modes and different break sizes in a transient. The training data are provided off-line from the test simulator. Major variables and the equipment status are combined for the input symptom vector in each transient
800
KWON, KIM, AND SEONG Table I. List of input vector variables. No.
Variable description
Unit
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Pressurizer pressure Pressurizer level Reactor coolant average temperature Steam generator pressure Steam generator level Reactor power Reactivity Average fuel temperature Feedwater line flow Main steam line flow Steam flow from steam generator Steam pressure from steam generator Secondary radiation monitoring Containment pressure Containment temperature Containment humidity Pressurizer relief tank pressure Pressurizer relief tank temperature Net electrical power Position of main steam isolation valve Reactor trip signal
kg/cm2 Normalized deg C kg/cm2 Normalized % % dk/k deg C m3 /hr m3 /hr m3 /hr kg/cm2 micro C/cc kg/cm2 deg C % kg/cm2 deg C MWe Normalized Digital
when the test simulator emulates a transient situation. The transients are simulated in the test simulator by activating the malfunctions during normal operation; then major variables are measured, such as temperature, pressure, flow, pump on/off status, or valve open/close status. The training data are collected from different operating modes, such as 50 percent, 55 percent, 60 percent, 65 percent, 70 percent, 75 percent, 80 percent, 85 percent, 90 percent, and 95 percent of reactor power, and full power and different break size in each transient and normal state. Each training datum consists of around 60 time interval input vectors. Each time interval is two seconds, so this means around two minutes of data are collected. The test data are collected off-line and on-line. The off-line test data are collected by the same method as training data, but they are collected for different operating modes and different break sizes to generate diverse test cases such as: trained operating modes and non-trained break sizes, non-trained operating modes and trained break sizes, non-trained operating modes and non-trained break sizes, and trained operating modes and trained break sizes. The on-line test data for any operating modes or break sizes are gathered directly from the real-time test simulator through data communication between the test simulator and the transient identification process. The test simulator is executed every 0.2 second, and the calculated simulation variables are stored in shared memory. The transient identification process receives on-line test data from shared memory every second. The data transfer scheme between three processes using shared memory is represented in Figure 3. The degree of validity of sensor signals proved to be a major factor in determining the accuracy of the diagnosis and the usefulness of the results. Most of the works
HIDDEN MARKOV MODEL-BASED REAL-TIME
801
Test simulator
Mathematical modeling process
Read/write (every 0.2 sec.) Read (every 1 sec.) Shared memory
Transient identification process
Supervisory process
Figure 3. Data transfer scheme between three processes using shared memory.
related to diagnosis applications assume that the sensor signals are always valid through the signal validation process. Sensor accuracy should be also considered in actual implementation of the transient identification system that would be applied to real NPPs. In this experimental environment, it can be assumed that all sensor signals are always valid and are free from interference such as sensor accuracy, because all the training and test data are from the test simulator. In order to verify the robustness of the developed system, an artificial sensor fault or equipment malfunction is activated in the experimental stage. In this implementation, we no longer consider the signal validation issue because signal validation is another important issue in this research arena. 3.4.
Real-Time Transient Identification System
The major components of the transient identification system are the vector quantizer and HMM identifier. Figure 4 shows the block diagram of the implemented transient identification system. First, the collected training data are normalized based on three normalized methods, then the training data are used to train the vector quantizer off-line. The trained vector quantizer will be used to vector quantize the test data. A vector-quantized codebook of training data is the training input set of the HMM identifier. The test symptoms are vector quantized to produce the input codebook of the HMM identifier. In this implementation, the K-means algorithm and SOM are used to cluster the input vector into L disjoint sets. In this implementation, the 300 clusters were chosen for an optimal solution after several attempts, meaning that every input vector is assigned to one of 300 clusters. The codebook size is 60, which means the system receives 60 time interval input vectors every minute to classify the types of
802
KWON, KIM, AND SEONG
HMM identifier
Vector quantizer
Input
Shared memory
Xt
Normalization
Test simulator
L-level codebook
Vector quantized observations
Output
Normal model
⌽t–k
Transient model 1
Identification
Transient description j(t)
HMM parameter re-estimation
Transient model 8 Transient model 9
Training TIS
Figure 4. A block diagram of transient identification system.
transients. In the initial minute, the test results are incorrect because the codebook size is less than 60. The system should wait until it receives a minute’s worth of input vectors. This initial stage is called the ready mode. During the next time step, the system receives another 60 time interval input vectors, as a result of the sliding window method. To achieve an input symptom vector every second, the system should be implemented with a real-time environment. It is difficult to implement an exact real-time environment under the UNIX operating system (OS) that is not a real-time OS. But it is possible to implement a loosely coupled real-time environment, which is enough for this transient identification implementation using the system call function provided by UNIX OS. A left-to-right HMM has been considered appropriate for processing those signals for which their properties change over time. The underlying state sequence associated with the model has the property that as time increases, the state index increases or stays in the same state. That is, the state always proceeds from left to right. The basic model consists of six states that have less than two direct transitions to the right state. Few initial conditions are given to this model, and these initial conditions are equivalent to all transient models. The re-estimation algorithm of the HMM might give a local minimum of the likelihood function. It is important to choose initial estimates of the HMM parameters so that the local minimum is the global minimum. Experience has shown that uniform initial estimates work well.30 We also choose uniform initial estimates in this implementation by assuming that the observation symbol probability is equivalent to each state. The training is performed by a forward algorithm and a backward algorithm, and then by making a re-estimate from the Baum-Welch algorithm in each model, given the multiple input observations.30 The re-estimation is done until the conver¯ ≥ P(O | λ), that is, the new model estimates are more gence condition, P(O | λ) likely to produce the given observation sequence O, is satisfied in each model. The
803
HIDDEN MARKOV MODEL-BASED REAL-TIME
probability, P(O | λ), is calculated by the optimal path, which is obtained by the Viterbi algorithm for the given input observations in each model. The transients are classified by examining which model has the highest probability for the given input observations. The prototype of the transient identification system was implemented using an HP747i industrial workstation, and the programming was done using the C language. 4.
EXPERIMENTAL RESULTS
In this section, experimental test results for a base model are described. Then, the selected model is suggested and the improved model is proposed to improve identification accuracy. The base model consists of a max-min normalization method, a SOM clustered by training and test data, and a six-state HMM identifier. The experimental tests have been carried out after training of the vector quantizer and HMM identifier. Table II shows the results of the base model off-line test. In this experiment, the following four test cases were used to compare the trained or nontrained data for operating modes and break sizes. • • • •
Case I: Trained operating mode, non-trained break size Case II: Non-trained operating mode, trained break size Case III: Non-trained operating mode, non-trained break size Case IV: Trained operating mode, trained break size
As presented in this table, most of the transients are correctly identified when given trained break size and non-trained operating mode. But in the case of nontrained break size, the recognition rate is remarkably lower than the trained break size. There is almost no difference when comparing the case of the trained and the non-trained operating mode. But the identification rate for the non-trained break size is lower than the non-trained operating mode. We estimated that the HMM can absorb these break size differences, but in real implementation, the HMM cannot completely absorb the break size differences. In Case IV, with both trained operating Table II. Off-line test results for base model. Recognition rates, % Transient
Case Case I
Case II
Case III
Case IV
Average Except Case IV
ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Average
— 100 27.3 45.5 — 0.0 72.7 100 100 63.6
100 100 100 100 100 100 80.0 100 100 97.8
— 100 30.0 50.0 — 0.0 70.0 100 100 64.3
100 100 100 100 100 100 100 100 100 100
100 100 51.6 64.5 100 32.3 74.2 100 100 76.8
804
KWON, KIM, AND SEONG 100
80 Identification rate (%)
ATWS FWLB LOCA
60
LSLC MSIV MSLI MSLO
40
PORV SGTR Avg.
20
0
0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 Time (seconds)
Figure 5. On-line identification rate for base model.
mode and trained break size, the identification rate is 100 percent. This means the trained data can be completely identified. The on-line test for the base model was performed using on-line test data that covers various operating modes and break sizes for each transient. The on-line test results are depicted in Figure 5. The total identification rate of the on-line test reaches 95.8 percent within 79 seconds, but the immediate detecting rate is only 44.3 percent. In the case of LOCA, MSLI, or PORV, the transients are correctly identified from incorrect transients. In the initial stage of these transients, there are no distinctive features between preceding patterns of identified and misidentified transients. The distinctive features appeared after a few seconds; then, the HMM identifier was able to identify the correct transient types. It takes some time to detect an SGTR transient, because its distinctive features only appear after some time in a real situation. The normalization methods are compared based on the assumption that the input normalization might influence the result of classification. The identification rate of each transient at different normalization methods is shown in Figure 6. As indicated in Figure 1, the normalization results of the mean-C1 method are not proper; the normalized data are concentrated near the mean. As a consequence of the normalization results of the mean-C1 method, the identification rate is lower than in the max-min or mean-C2 methods. The identification rate of max-min is higher than the other two methods. The max-min method has advantages in the present case. Consequently, the max-min method is chosen for the selected model. To choose the proper clustering algorithm, two types of clustering methods are compared, and the results are depicted in Figure 7. Considering the results, the K-means algorithm has better performance than the SOM. Finally, the K-means algorithm method with training data is chosen to implement the selected model.
805
HIDDEN MARKOV MODEL-BASED REAL-TIME
100 ATWS
Identification rate (%)
80
FWLB LOCA LSLC
60
MSIV MSLI MSLO
40
PORV SGTR
20
Avg.
0 max-min
mean-C1 Normalization methods
mean-C2
Figure 6. Identification rate based on normalization methods.
Identification rate (%)
100 ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Avg.
80 60 40 20 0 SOM
K-means Clustering methods
Figure 7. Identification rate based on clustering methods.
The selected model is suggested according to the results of the previous experimental tests. The selected model means adopting a proper normalization method, clustering algorithm, and number of HMM states. According to the results of previous experimental tests, the selected model has a max-min normalization method, K-means clustering algorithm, and six states of left-to-right HMM. The total identification rate of the selected model is 100 percent within 17 seconds, and its immediate detection rate is 54.0 percent. The total identification and immediate detection rate of the selected model is higher than the base model. In the selected model, the detection rate of correct transients from incorrect transients is significantly reduced from the base model. According to the results of the above experimental tests, it can
806
KWON, KIM, AND SEONG 100 90
Identification rate (%)
80
ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Avg.
70 60 50 40 30 20 10 0 0
2
4
6
8 10 12 Random noise (%)
14
16
18
20
Figure 8. Test results of superimposing random noise.
be concluded that the selected model has good performance, but it cannot be said that the selected model is robust. Therefore, three more experimental tests were performed, including superimposing random noise, adding systematic error, and testing by untrained transients to verify the robustness of the selected model. The random noise is superimposed on the original test data, except for two variables, such as position of the main steam isolation valve and reactor trip signal. The terminology “2 percent of random noise” means a maximum ±2 percent of random noise is superimposed on test data in the normalizing progress. The test results of superimposing random noise are depicted in Figure 8. In the case of 2 percent to 8 percent of random noise, the identification rates are the same as the no random noise case. The identification rate decreases slightly at 10 percent of random noise. But, the identification rate abruptly decreases for more than 12 percent of random noise. It can be said that the proposed transient identification system is robust within 10 percent of random noise. In neural network applications, the recognition rate linearly decreases by increasing the noise level,2 or the recognition rate is maintained to 16 percent of the noise level, then the rate abruptly decreases.10 Additional tests were performed in which the systematic error signal is added differently from the random noise. The first error signal is a “loss of pressurizerlevel signal,” which has large variations when the transients occur. In this case, the identification rate is significantly lower than with no error signal. From this test result, to ensure the performance of the transient identification system, valid signals through a signal validation process should be provided. In the application of a neural network case, the system can identify the transient properly even though some sensor signals are missed.36 But the second error signal is a “loss of main steam flow signal,” which has small variations when the transients occur. In this case, the identification
807
HIDDEN MARKOV MODEL-BASED REAL-TIME
100 ATWS
Identification rate (%)
80
FWLB LOCA LSLC
60
MSIV MSLI
40
MSLO PORV SGTR
20
Avg.
0 No error
Loss of pressurizerlevel signal
Loss of main steam flow signal
Systematic error
Figure 9. Test results of adding systematic error.
rate is the same as if there was no error signal. This means the error signals that have small variations have almost no effect on identification rate. The test results of adding a systematic error are depicted in Figure 9. It is desirable that the never-trained transient should be classified as an unknown transient. In particular, a severe accident can happen because of an inadequate operation due to incorrect identification. The implemented system can classify the unknown transient similar to the other neural network applications.3,9,16 A transient is classified as unknown if the output path probability of the HMM identifier is less than the threshold that has the least output path probability of all test data. The test results of untrained transients are summarized in Table III. As shown in Table III, three untrained transients are classified as unknown transients. But four untrained transients are identified as normal states. In the case of the incorrect classification, the HMM identifier might not be exactly identified, because the clustering results are very similar to the misclassified transient. In the selected model, there still exist temporary misclassified transients. Finally, two heuristic training approaches are attempted to improve the classification Table III. Test results of untrained transient. Untrained Transient Description Drop of all control rods in control bank group A Reactor coolant pump trip Charging line rupture Turbine control valve failure (close) Steam line rupture (100 cm2 ) Failure in feedwater controller (loss of steam flow signal) Failure in pressurizer controller: maximum failure
Reactor Power (%)
Test Result
95 88 78 90 100 100 95
Unknown Unknown Normal state Unknown Normal state Normal state Normal state
808
KWON, KIM, AND SEONG 100
Identification rate (%)
90
80 Base model
70
Selected model Improved model
60
50
40
0
20
40
60
80
Time (seconds) Figure 10. On-line identification rate for three models.
accuracy. One of these approaches is corrective training.37 The heuristic corrective training method re-estimates model parameters using a validation data set that is not included in the training or test data sets. After the re-estimation of HMM parameters using the training data set, transient identification is performed on the validation data. If any transient is misclassified, the estimated model parameters are adjusted to reduce the probability of misclassified transients. In real implementation, the corrective training is applied to only ATWS and FWLB transients to improve the immediate detection rate. The other heuristic approach is the principal component method. The principal component is applied to increase discriminating power between two transients that are expressed by similar patterns, such as FWLB and LOCA transients. This method adds a weighting factor to important variables that have large impacts on identification results when vector quantization is progressed. The secondary radiation variable is chosen as an important variable to differentiate FWLB and LOCA transients in this implementation. Consequently, the classification rate is slightly increased when the principal component method is applied. In conclusion, Figure 10 shows the classification rate of three on-line tests over time. The classification rate of the selected model is much better than the base model, and the classification rate of the improved model is a little better than the selected model. 5.
CONCLUSIONS
The proposed transient identification system has many advantages, as described above. However, there are still some problems that should be solved before it can actually be applied to operating NPPs. Further efforts are being made to improve the
HIDDEN MARKOV MODEL-BASED REAL-TIME
809
system performance and robustness to demonstrate reliability and accuracy to the required level. Following are the identified problems and suggestions learned from this work. It is a major problem that the training data are extremely rare compared with other applications. The HMM-based identifier is more difficult to train properly because it tends to require more training data. For this reason, research on a modified HMM structure or a new algorithm for the estimation of HMM parameters is suggested. As shown in experimental tests, the identification performance heavily depends on clustering methods. Therefore it is very important to find more efficient clustering methods that are suitable for the proposed system. The HMM-based identifier cannot completely resolve the “don’t know” issue. Future work should solve this problem. A fitness measure also needs to be developed to confirm the degree of belief. Besides the improvements described in the improved model, it needs to be integrated with other features such as knowledge processing38 or neural networks 22 to improve its accuracy. Artificial intelligence (AI) techniques such as neural networks, fuzzy logic, and HMM that are applicable to NPPs require careful consideration.39 In particular, AIbased, safety-critical systems should be treated with caution and need confirmation from the regulatory body. Acknowledgment The authors wish to thank the anonymous reviewers for their useful comments and corrections. This work has been carried out under the Nuclear Research and Development Program by the Ministry of Science and Technology in Korea.
References 1. 2. 3.
4. 5. 6. 7. 8. 9.
Kim IS. Computerized systems for on-line management of failures: A state-of-the-art discussion of alarm and diagnostic systems applied in the nuclear industry. Reliability Eng Syst Safety 1994;44:279–295. Ikonomopoulos A, Uhrig RE, Tsoukalas L. A hybrid neural network-fuzzy logic approach to nuclear power plant transient identification. AI91 Frontiers in Innovative Computing for the Nuclear Industry. Jackson, WY; Sep 15–18, 1991. pp 217–226. Bartal Y, Lin J, Uhrig R. Transients identification in nuclear power plants using probabilistic neural networks and the problem of knowledge extrapolation. In: 9th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 24–26, 1995. pp 49.01– 49.08. U.S. Nuclear Regulatory Commission. Functional criteria for emergency response facilities, NUREG-0696; Feb 1981. Bruckner LA. On Chernoff faces. In: Wang PCC, editor. Graphical representation of multivariate data. New York: Academic Press; 1978. U.S. Nuclear Regulatory Commission. Human factors acceptance criteria for the safety parameter display system, NUREG-0835; Oct 1981. Guo Z, Uhrig RE. Accident scenario diagnostics with neural networks. In: 8th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 1992. pp 53.01–53.11. Basu A. Nuclear power plant status diagnostics using a neural network with dynamic node architecture. MS thesis, Iowa State University; 1992. Bartal Y, Lin J, Uhrig R. Nuclear power plant transient diagnostics using artificial neural networks that allow “don’t-know” classifications. Nucl Technol 1995;110:436–449.
810 10. 11. 12. 13. 14. 15.
16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33.
KWON, KIM, AND SEONG Jeong E, Furuta K, Kondo S. Identification of transient in nuclear power plant using neural network with implicit time measure. In: Proc Topical Meeting on Computer-Based Human Support Systems. Philadelphia, PA; June 25–29, 1995. pp 467–474. Parlos AG, Atiya AF, Chong KT, Tsai WK. Nonlinear identification of process dynamics using neural networks. Nucl Technol 1992;97:79–96. Roh CH, Chang HS, Kim HG, Chang SH. Identification of reactor vessel failures using spatiotemporal neural networks. IEEE Trans Nucl Sci 1996;43(6):3223–3229. Iijima T. Application study of fuzzy logic method for plant-state identification. HWR-432, OECD Halden Reactor Project; Dec 1995. Lin J, Bartal Y, Uhrig R. Using similarity based formulas and genetic algorithms to predict the severity of nuclear power plant transients. In: Proc 9th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 24–26, 1995. pp 53.01–53.09. Jeong E, Furuta K, Kondo S. Identification of transient in nuclear power plant using adaptive template matching with neural network. In: Nuclear Plant Instrumentation, Control, and Human-Machine Interface Technologies. Pennsylvania State University; May 6–9, 1996. pp 243–250. Hines JW, Miller DW, Hajek BK. A hybrid approach for detecting and isolating faults in nuclear power plant interacting systems. Nucl Technol 1996;115:342–358. Zhu Q. Hidden Markov model for dynamic obstacle avoidance of mobile robot navigation. IEEE Trans Robot Automat 1991;7(3):390–397. DeWitt MR. High range resolution radar target identification using the PRONY model and hidden Markov models. MS thesis, Air Force Institute of Technology, WPAFB OH; Dec 1992. Yamato J, Ohya J, Ishii K. Recognizing human action in time-sequential images using hidden Markov model. In: Proc Computer Vision and Pattern Recognition. Urbana-Champaign, IL; 1992. pp 379–385. Starner TE. Visual recognition of American sign language using hidden Markov models. MS thesis, MIT; Jan 1995. Thoraval L, Carrault G, Bellanger J. Heart signal recognition by hidden Markov models: The ECG case. Methods Info Med 1994;33(1):10–14. Kundu A, Chen G. An integrated hybrid neural network and hidden Markov model classifier for sonar signal classification. ICASSP 1995;3587–3590. Brand M, Oliver N, Pentland A. Coupled hidden Markov models for complex action recognition, CVPR97. San Juan, Puerto Rico; June 17–19, 1997. pp 994–999. Hatzipantelis E, Penman J. The use of hidden Markov models for condition monitoring electrical machines. In: 6th Int Conf on Electrical Machines and Drives. Sept 8–10, 1993. Smyth P. Hidden Markov models for fault detection in dynamic systems. Pattern Recognit 1994;27(1):149–164. Fielding KH, Ruck DW. Recognition of moving light displays using hidden Markov models. Pattern Recognit 1995;28(9):1415–1421. Couvreur C, Fontaine V, Gaunard P, Mubikangiey G. Automatic classification of environmental noise events by hidden Markov models. Appl Acoustics 1998;54(3):187–206. Kulp D, Haussler D, Reese MG, Eeckman FH. A generalized hidden Markov for the recognition of human genes in DNA. In: Proc 4th Int Conf on Intelligent Systems for Molecular Biology. Washington University, St. Louis; June 12–15, 1996. pp 134–142. Smyth P. Tutorial material; machine learning: Theory and application. In: Proc 3rd World Congress on Expert Systems, Seoul. Korea; Feb 5–9, 1996. Huang XD, Ariki Y, Jack MA. Hidden Markov models for speech recognition. Edinburgh: Edinburgh University Press; 1990. Couvreur C. Environmental sound recognition: A statistical approach. Ph.D. thesis, Facult´e Polytechnique de Mons, Belgium; June 1997. Rabiner LR. A tutorial on hidden Markov models and selected application in speech recognition. Proc IEEE 1989;77(2):257–285. Kozma R, Yokoyama Y, Kitamura M. Intelligent monitoring of NPP anomalies by adaptive neuro-fuzzy signal processing system. In: Proc Topical Meeting on Computer-Based Human Support Systems. Philadelphia, PA; June 25–29, 1995. pp 449–456.
HIDDEN MARKOV MODEL-BASED REAL-TIME
811
34. Kohonen T. The self-organizing map. Proc IEEE 1990;78(9):1464–1480. 35. Kwon KC, Song SJ, Park WM, Lyu SP. The real-time functional test facility for advanced instrumentation and control in nuclear power plants. IEEE Trans Nucl Sci 1999;46(2). 36. Cheon SW, Chang SH. Application of neural networks to a connectionist expert system for transient identification in nuclear power plants. Nucl Technol 1993;102:177–191. 37. Applebaum T, Hanson B. Enhancing the discrimination of speaker independent hidden Markov models with corrective training. In: Proc ICASSP-89. Glasgow, Scotland; May 1989. pp 302–305. 38. Ohga Y, Seki H. Abnormal event identification in nuclear power plants using a neural network and knowledge processing. Nucl Technol 1993;101:159–167. 39. Wainwright N. A regulator’s viewpoint on the use of AI in the nuclear industry. Nucl Energy 1995;34(2):93–97.
In this article, a transient identification method based on a stochastic approach with the hidden Markov model (HMM) has been suggested and evaluated experimentally for the classification of nine types of transients in nuclear power plants (NPPs). A transient is defined as when a plant proceeds to an abnormal state from a normal state. Identification of the types of transients during an early accident stage in NPPs is crucial for proper action selection. The transient can be identified by its unique time-dependent patterns related to the principal variables. The HMM, a double-stochastic process, can be applied to transient identification that is a spatial and temporal classification problem under a statistical pattern-recognition framework. The trained HMM is created for each transient from a set of training data by the maximum-likelihood estimation method which uses a forward-backward algorithm and the Baum-Welch re-estimation algorithm. The transient identification is determined by calculating which model has the highest probability for given test data using the Viterbi algorithm. Several experimental tests have been performed with normalization methods, clustering algorithms, and a number of states in HMM. There are also a few experimental tests that have been performed, including superimposing random noise, adding systematic error, and adding untrained transients to verify its performance and robustness. The proposed real-time transient identification system has been proven to have many advantages, although there are still some problems that should be solved before applying it to an operating NPP. Further efforts are being made to improve the system performance and robustness in order to demonstrate reliability and accuracy to the required level. © 2002 Wiley Periodicals, Inc.
1.
INTRODUCTION
Transient identification in nuclear power plants means classifying the types of transients by interpreting the major plant variables and operating status of the equipment. A transient is defined as when a plant proceeds to an abnormal state from * Author to whom all correspondence should be addressed; e-mail: [email protected]. INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, VOL. 17, 791–811 (2002) © 2002 Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.002/int.10050
792
KWON, KIM, AND SEONG
a normal state. If the error, either caused by a human or plant component, happens to be followed by a fault, then a transient has occurred. Consequently, an accident is preceded by a transient. The term identification, as applied to an engineering system or process, means the classification of the cause that brought either an undesirable state or failure of the system. The identification can be done at several different levels, for example, component, subsystem, function, or event.1 For the proposed transient identification system, identification is made at the event level to determine which transient has occurred in the NPP. It is necessary to identify the type of transient through continuous monitoring of the NPP as a complex dynamic system during its early stage to provide sufficient information to the human operators, in order to assist in proper operator action selection to prevent a more severe situation or to mitigate the accident consequence.2 Also, this system provides input for computerized accident management, computerized operating procedure, and semiautonomous operation systems. Typical transients in NPPs are associated with unique, time-dependent patterns of major variables and equipment status. This time-dependent pattern can be used to identify the transient; hence, identification can be treated as a pattern classification problem.3 The initial trial to identify plant status using pattern classification was the introduction of the safety parameter display system (SPDS) after the Three Mile Island (TMI) accident. It follows the U.S. Nuclear Regulatory Commission (USNRC) Safety Guide NUREG-0696 philosophy.4 It monitors critical plant parameters and concentrates the information in such a way as to give a systematic view of the safety status of the plant, especially under accident conditions. It also assists the operator in his or her diagnosis and decision-making. An example of SPDS is the spider’s webstyle display: the human operator tries to identify the plant’s status by utilizing the unique patterns of parameters from a spider’s web-style display. Another example of SPDS is the Chernoff face, a graphic technique that maps multidimensional data into facial features. The most common usages of the technique are to display the data in a convenient form, to aid in discovering clusters and outliers, and to show changes over time.5,6 In both of the above cases, the displays are provided by machine, but classification is carried out by a human operator. The human operator instantaneously uses only static spatial information, not temporal information. It is difficult for a human operator to identify a transient when the preceding patterns of some transients are very similar and the patterns change further with time. Recently, attempts have been made to solve transient identification problems using a computer-based system. In this transient identification problem, the classification might involve spatial and temporal patterns. Temporal patterns usually involve ordered sequences of data appearing over time. Spatial patterns mean the unique patterns of each transient, variations of the same transient that might occur under different operating modes or at different break sizes. The transient identification systems for NPPs have been developed using techniques such as artificial neural networks (ANNs) and fuzzy logic. Initially, the simplest feedforward backpropagation neural network (NN) techniques were used. Later, many attempts were made to improve identification performance using ANNs to discriminate similar transients,2 reduce the number of training patterns using lateral feedback NNs,7 incorporate modular hierarchy with dynamic node architecture
HIDDEN MARKOV MODEL-BASED REAL-TIME
793
to reduce the size of the system,8 and use probabilistic NN,3 including the “don’t know” answer.9 Simple feedforward NN applications that do not consider the time concept still cannot adequately solve the temporal variations. Recently, there have been many applications that include a time concept such as NN with an implicit time measure,10 recurrent multi-layer perceptron,11 and spatiotemporal NN.12 There are also numerous applications other than ANN, such as fuzzy logic,13 nearest neighbors modeling optimized by using a genetic algorithm,14 adaptive template matching,15 and observer-based residual generation.16 All of these systems are still considered as prototypes or are under evaluation and have not yet been applied to real, operating NPPs. The ANN and fuzzy logic approach can absorb spatial variations, but cannot provide proper solutions for temporal variations. Thus, it is reasonable to adopt a double-stochastic approach for the classification of the patterns. The hidden Markov model (HMM), a double-stochastic process, enables modeling of not only spatial phenomena, but also time-scale distances. The HMM can be used to solve classification problems associated with time series input data such as speech signals or plant process signals, and can provide appropriate solutions by its modeling and learning capabilities, even though it does not have the exact knowledge to solve the problems. Most of the HMM applications for pattern classification in dynamic processes have a typical architecture to solve spatial-temporal problems, but the target systems are different, as in dynamic obstacle avoidance of mobile robot navigation,17 radar target,18 human action,19 American sign language,20 heart signals,21 sonar signals,22 two-handed actions,23 conditions of an electrical machine,24 deep space network antennae,25 moving light displays,26 environmental noise,27 and human genes in DNA.28 But the HMM has never been applied for transient identifications in NPPs. The goal of this article is to provide a real-time transient identification system for use in NPPs that demonstrates the spatial and temporal modeling and learning capabilities with hidden Markov models. To demonstrate these capabilities, simulated NPP parameters are collected and clustered to build hidden Markov models for transient identification purposes. 2.
HIDDEN MARKOV MODEL FOR TRANSIENT IDENTIFICATION
The problem of transient identification is defined as the recognition of transient types, ω j , given the sequential input patterns, X t , at time t. The input pattern, X t , is mathematically defined as an object described by a sequence of features at time t 29 : X t = (x1 , x2 , . . . , xi , . . . , xd )
(1)
The space of input pattern X t consists of the set of all possible patterns: X t ⊂ d ,
where d is a d-dimensional real vector space.
The k observed data up to time t are defined as t−k : t−k = {X t−k+1 , . . . , X t−1 , X t }
(2)
794
KWON, KIM, AND SEONG
The set of possible transient classes, ω j , at time t forms the space of classes, : = {ω1 , ω2 , . . . , ωc },
where c is the number of classes
(3)
The classes, , are assumed to be mutually exclusive and exhaustive. The recognition task can be considered to be the finding of function f , which maps the space of input patterns, t−k , to the space of classes, : f : t−k →
(4)
A dynamic process often exhibits sequentially changing behavior. If the one short time period is defined as a frame, the probability of the frame transition is different from each transient in NPPs. Therefore, the probability of frame existence and the transition between frames can be statistically modeled. The probability of a transient occurring in an NPP is already given and is called the a priori probability. When a transient has occurred in NPPs, the type of transient can be determined only by selecting the type of transient, ω j , with the highest a priori probability, P(ω j ). This decision is obviously unreasonable. It is more reasonable to determine the type of transient after observing the trend of time series major variables, namely, to get the conditional probability, P(ω j | t−k ). This conditional probability is called the a posteriori probability. Decision-making based on the a posteriori probability is more reliable, because it employs both a priori knowledge together with observed time-series data.30 Classification of the unknown pattern, X t , corresponds to finding the optimal model, ω, ˆ that maximizes the conditional probability, P(ω j | t−k ), the probability that the system is in class ω j at time t given that t−k was observed at time t. We can apply Bayes’ rule to calculate the a posteriori probability, P(ωˆ | t−k ) = max ω
P(t−k | ω j )P(ω j ) P(t−k )
(5)
where P(t−k ) =
c
P(t−k | ω j )P(ω j )
(6)
j=1
The conditional probability, observing t−k given that the system is in class ω j at time t, P(t−k | ω j ), comes from comparing the shapes of the transient models with input observations, while the a priori probability, P(ω j ), comes from the transient probability that represents how often the transient appears in the NPP. Because ˆ we get: P(t−k ) is independent of ω, ˆ ω) ˆ P(ωˆ | t−k ) ∝ P(t−k | ω)P( = max[P(t−k | ω j )P(ω j )] ω
(7)
In fact, the a priori probability, P(ω j ), can be calculated in NPPs, and should satisfy the following condition: c j=1
P(ω j ) = 1
(8)
HIDDEN MARKOV MODEL-BASED REAL-TIME
795
But the transient identification system does not cover all of the transients occurring in NPPs, and consequently cannot satisfy Equation 8. Therefore, the present observed data controls the decision. P(t−k | ω j ) is called the likelihood of ωˆ with respect to the set of samples. In real implementation, the a priori probability, P(ω j ), can be assumed that the occurring probabilities of all transients are equal.9 The maximum likelihood estimate of ωˆ is, by definition, that value of ωˆ that maximizes P(t−k | ω j ). The HMM can successfully treat a transient identification problem under a probabilistic or statistical framework.30 The HMM is nothing more than a specific type of parametric probability distribution model. Therefore, all the statistical pattern recognition principles that have been presented above can be applied to HMMs.31 In this identification problem, the HMM is used to estimate the conditional probability, P(t−k | ω j ). By using HMM, the pattern variability in parameter space and time can be modeled effectively. The HMM uses a Markov chain to model changing statistical characteristics that exist in the actual observations of dynamic process signals. The Markov process is therefore a double-stochastic process in which there is an unobservable Markov chain defined by a state transition matrix, and where each state of the Markov chain is associated with a discrete output probability distribution. The double-stochastic processes enable modeling of not only spatial phenomena, but also time-scale distances. Using HMMs in a classification problem with Bayes’ rule and maximum likelihood training requires two things: the evaluation of P(t−k | ω j ) for the implementation of Bayes’ rule, and the maximization of the likelihood for the training of the classifiers. Fortunately, there exist computationally efficient procedures for these two tasks. HMM parameters are estimated from the Baum-Welch algorithm and guarantee a finite improvement on each iteration in the sense of maximization of likelihood. An HMM is trained for each transient from a set of training data and an iterative maximum likelihood estimation of model parameters from observed timeseries data. Incoming observations are classified by calculating which model has the highest probability of producing that observation. An HMM can be represented by the compact notation λ = (A, B, ). Specification of an HMM involves choosing the number of states, N , the number of discrete symbols, L, and the specification of three probability densities with matrix form, A, B, and . The more detailed definition of HMM is described in Ref. 32. Training refers to the characteristics of input patterns to be modeled by the parameter λ = (A, B, ). In this article, the maximum likelihood estimation is used for training. This maximum likelihood estimation of the parameters of a hidden Markov process requires as many observations as possible in a practical implementation. However, in real applications, only finite training data are available, and in some parameters are inadequately trained. Recognition or classification means finding the best path in each trained model and selecting the one that maximizes the path probability for a given input observation. Therefore, given the observation O = O1 , O2 , . . . , OT and the model λi = (Ai , Bi , i ), where i = 1, 2, . . . , C, the classified model λ∗ satisfies the following equation: λ∗ = max P(O | λi ), i
where 1 ≤ i ≤ C
(9)
796
KWON, KIM, AND SEONG
3.
TRANSIENT IDENTIFICATION SYSTEM 3.1.
Pre-processing
Training and test data from the test simulator should be converted to a proper codebook that is input data for the HMM identifier. Pre-processing means to make a codebook, and this method is divided into two techniques: normalization and vector quantization. The input symptom vector to be used for vector quantization has the same value as displayed in a real plant instrument, because it is directly received from the test simulator or the plant computer in NPPs. The range of input data values is significantly different. Therefore, it should be normalized before using the input data of the clustering algorithms. Normalization is one of several transformation techniques. It has the effect of reducing the parameters to a common range. If the distribution of the variable’s values are of normal distribution, it is reasonable to be normalized by maximum value. But as shown in Figure 1, most of the values of the specified variable are distributed in the upper region in the case of normalization by maximum value. It is undesirable to use the input data of the clustering algorithms. It is more reasonable to normalize between its minimum and maximum values, as in the following equation33 : Normalized value (1) = (Value − Min.)/(Max. − Min.)
(10)
In a similar way, mean-centered normalization can be using the following equation: Normalized value (2) = ((Value − Mean)/(0.5 × Range of values) + 1.0) × 0.5 (11) In the mean-centered normalization method, data between the mean and minimum or maximum is linearly distributed. Alternatively, we can define a nonlinear data distribution between the mean and the minimum or maximum range considering the standard deviation using the following equation: Normalized value (3) = ((Value − Mean)/(Cσ ) + 1.0) × 0.5
(12)
C is the constant from the range of values and standard deviation. The results of normalization of the above four methods regarding a specific variable, pressurizer pressure, are shown in Figure 1. In Figure 1, Mean-C1 data are from Equation 11 and Mean-C2 data are from Equation 12. Features can be represented by continuous, discrete, or discrete-binary variables. It is expected that the feature vector contains most of the classification information available from the object.31 Feature extraction is an important task for classification or recognition, and is often necessary as a pre-processing stage for data. In feature extraction, data can be transformed from high-dimensional pattern space to low-dimensional feature space. Vector quantization (VQ), the process of approximating a block of continuous amplitude signals by a discrete signal, is one method of feature extraction. The idea is to quantize each continuous vector to one of a relatively small number of template vectors, which together comprise what is called a codebook. The sequence of codebook indices obtained in this way forms the
797
HIDDEN MARKOV MODEL-BASED REAL-TIME 3,500 max-0
3,000
mean-C1 mean-C2
Distribution
2,500 2,000 1,500 1,000 500 0 0.0
0.2
0.4 0.6 Normalized value
0.8
1.0
Figure 1. Comparison of normalization methods.
desired sequence of discrete symbols. In this article, two VQ methods are introduced to compare the identification capability. To minimize iteratively the average distortion measure, the most widely used method is the K-means algorithm. In the K-means algorithm, the basic idea is to divide the set of training vectors into L clusters, Ci , {1 ≤ i ≤ L}, in such a way that two necessary conditions for optimality are satisfied. The first condition is that the optimal quantization is realized by using a nearest neighbor selection rule. The second condition is that each codeword is chosen to minimize the average distortion.30 The feature mapping algorithm is supposed to convert patterns of arbitrary dimensionality into the response of a one-dimensional array of neurons. Suppose that an input pattern has N features and is represented by a vector x in an n-dimensional pattern space. The network maps the input patterns to an output space. The output space in this case is assumed to be a one-dimensional array of output nodes, which possess a certain topological orderliness. The question is how to train a network so that the ordered relationship can be preserved. Kohonen proposed allowing the output nodes to interact laterally, leading to the self-organizing map (SOM), also known as the Kohonen network.34 3.2.
Real-Time Test Environment
It is more realistic to receive training and test data directly from the operating NPPs. But severe transients or accidents seldom occur in real NPPs, and it is almost impossible to create transient conditions only for experimental purposes. Therefore, it was necessary to use a simulator for operator training, or simulation code ready for safety analysis, to implement and test the transient identification system. In this
798
KWON, KIM, AND SEONG
Modeling program
Real-time executive
Static program
Dynamic program
Shared memory
Print-out selected variables
RUN/FREEZE/ONE_STEP
Get variable value
Command interpreter
Store global variables
Change variable value
Get global variables
Display variables value
Malfunction insert Supervisory program
Figure 2. Software structure of test simulator.
implementation, the test simulator was modified from the compact nuclear simulator, installed at the Korea Atomic Energy Research Institute Nuclear Training Center for training non-operator personnel, for testing the transient identification system. The test simulator is divided into two major parts: a mathematical modeling program, which executes the plant dynamic modeling program in real-time, and a supervisory program that manages user instructions. The simplified software structure of the test simulator is depicted in Figure 2. The NPP modeled in the test simulator is a three-loop, 993 MWe Westinghouse pressurized water reactor, mostly used in the Kori Units 3 and 4 in Korea. The mathematical modeling programs consist of static and dynamic parts. The initial state, a 100 percent full-power condition is set up in the static calculation, which is performed once before the start of the dynamic calculation. The dynamic calculation is performed every 0.2 second to represent a real-time mathematical modeling simulation. The test simulator provides the function to activate 79 predefined malfunctions. This function realizes the transient or accident condition to get training data and to test the transient identification system.35 3.3.
Data Collection
The transients or accidents in NPPs can be categorized by the required operator response, the required plant protection system actuation, expected frequency of occurrence, or postulated initiating event. The nine typical transients are selected
HIDDEN MARKOV MODEL-BASED REAL-TIME
799
among different postulated transients that might occur in NPPs considering the simulation capability of the test simulator, because the purpose of this article is to demonstrate the capability of hidden Markov models that apply to transient identification problems. The detailed descriptions of the target transients are as follows: (1) ATWS (anticipated transient without scram) The scram signal is lost, caused by a reactor protection logic circuit failure at any power level. The control rods are not moving down on the scram signal. (2) FWLB (feedwater line break inside containment) The feedwater line is broken inside containment. (3) LOCA (small loss-of-coolant accident) A leakage of primary coolant into containment. A break is assumed in the primary loop as a partial rupture of the pipe itself or as a rupture of the smaller pipes connected into the primary loop. (4) LSLC (loss of steam generator-level controller signal) Steam generator level control signal is lost caused by steam generator level controller failure. (5) MSIV (main steam isolation valve closure) The main steam isolation valve is closed, caused by a driving mechanism failure. (6) MSLI (main steam line break inside containment) The whole main steam line pipe or a part of it is broken. Because the secondary isolation valves are located in the turbine hall, the possible combination is nonisolable leak inside containment. (7) MSLO (main steam line break outside containment) The same as MSLI, when the possible combination is nonisolable leak outside containment. (8) PORV (power-operated relief valve stuck open) The pressurizer power-operated relief valve is stuck open, caused by control mechanism failures. (9) SGTR (steam generator tube rupture) A rupture of the steam generator tubes. As a consequence of thermal stresses on the tube material, one or several steam generator tubes are broken, opening a flow connection between the primary and secondary loops.
The input symptom vector is a collection of the principal variables and the status of major equipment from the transient simulation in the test simulator. The major variables and equipment status used to identify the nine different types of transients and one normal state are summarized in Table I. The selection of variables is dependent on the knowledge of an experienced reactor operator and accident analysis from the safety analysis report (SAR). There are two types of data, training data and test data. The training data are needed to train the clustering algorithm and HMM identifier. To test the classification capability of the HMM identifier, test data are also needed. The training and test data are collected from the test simulator. It is needed to get more widely spread training data per transient to design a reasonable classifier. In this experiment, we considered different operating modes and different break sizes in each transient. The HMM might absorb the variations from the different operating modes and different break sizes in a transient. The training data are provided off-line from the test simulator. Major variables and the equipment status are combined for the input symptom vector in each transient
800
KWON, KIM, AND SEONG Table I. List of input vector variables. No.
Variable description
Unit
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Pressurizer pressure Pressurizer level Reactor coolant average temperature Steam generator pressure Steam generator level Reactor power Reactivity Average fuel temperature Feedwater line flow Main steam line flow Steam flow from steam generator Steam pressure from steam generator Secondary radiation monitoring Containment pressure Containment temperature Containment humidity Pressurizer relief tank pressure Pressurizer relief tank temperature Net electrical power Position of main steam isolation valve Reactor trip signal
kg/cm2 Normalized deg C kg/cm2 Normalized % % dk/k deg C m3 /hr m3 /hr m3 /hr kg/cm2 micro C/cc kg/cm2 deg C % kg/cm2 deg C MWe Normalized Digital
when the test simulator emulates a transient situation. The transients are simulated in the test simulator by activating the malfunctions during normal operation; then major variables are measured, such as temperature, pressure, flow, pump on/off status, or valve open/close status. The training data are collected from different operating modes, such as 50 percent, 55 percent, 60 percent, 65 percent, 70 percent, 75 percent, 80 percent, 85 percent, 90 percent, and 95 percent of reactor power, and full power and different break size in each transient and normal state. Each training datum consists of around 60 time interval input vectors. Each time interval is two seconds, so this means around two minutes of data are collected. The test data are collected off-line and on-line. The off-line test data are collected by the same method as training data, but they are collected for different operating modes and different break sizes to generate diverse test cases such as: trained operating modes and non-trained break sizes, non-trained operating modes and trained break sizes, non-trained operating modes and non-trained break sizes, and trained operating modes and trained break sizes. The on-line test data for any operating modes or break sizes are gathered directly from the real-time test simulator through data communication between the test simulator and the transient identification process. The test simulator is executed every 0.2 second, and the calculated simulation variables are stored in shared memory. The transient identification process receives on-line test data from shared memory every second. The data transfer scheme between three processes using shared memory is represented in Figure 3. The degree of validity of sensor signals proved to be a major factor in determining the accuracy of the diagnosis and the usefulness of the results. Most of the works
HIDDEN MARKOV MODEL-BASED REAL-TIME
801
Test simulator
Mathematical modeling process
Read/write (every 0.2 sec.) Read (every 1 sec.) Shared memory
Transient identification process
Supervisory process
Figure 3. Data transfer scheme between three processes using shared memory.
related to diagnosis applications assume that the sensor signals are always valid through the signal validation process. Sensor accuracy should be also considered in actual implementation of the transient identification system that would be applied to real NPPs. In this experimental environment, it can be assumed that all sensor signals are always valid and are free from interference such as sensor accuracy, because all the training and test data are from the test simulator. In order to verify the robustness of the developed system, an artificial sensor fault or equipment malfunction is activated in the experimental stage. In this implementation, we no longer consider the signal validation issue because signal validation is another important issue in this research arena. 3.4.
Real-Time Transient Identification System
The major components of the transient identification system are the vector quantizer and HMM identifier. Figure 4 shows the block diagram of the implemented transient identification system. First, the collected training data are normalized based on three normalized methods, then the training data are used to train the vector quantizer off-line. The trained vector quantizer will be used to vector quantize the test data. A vector-quantized codebook of training data is the training input set of the HMM identifier. The test symptoms are vector quantized to produce the input codebook of the HMM identifier. In this implementation, the K-means algorithm and SOM are used to cluster the input vector into L disjoint sets. In this implementation, the 300 clusters were chosen for an optimal solution after several attempts, meaning that every input vector is assigned to one of 300 clusters. The codebook size is 60, which means the system receives 60 time interval input vectors every minute to classify the types of
802
KWON, KIM, AND SEONG
HMM identifier
Vector quantizer
Input
Shared memory
Xt
Normalization
Test simulator
L-level codebook
Vector quantized observations
Output
Normal model
⌽t–k
Transient model 1
Identification
Transient description j(t)
HMM parameter re-estimation
Transient model 8 Transient model 9
Training TIS
Figure 4. A block diagram of transient identification system.
transients. In the initial minute, the test results are incorrect because the codebook size is less than 60. The system should wait until it receives a minute’s worth of input vectors. This initial stage is called the ready mode. During the next time step, the system receives another 60 time interval input vectors, as a result of the sliding window method. To achieve an input symptom vector every second, the system should be implemented with a real-time environment. It is difficult to implement an exact real-time environment under the UNIX operating system (OS) that is not a real-time OS. But it is possible to implement a loosely coupled real-time environment, which is enough for this transient identification implementation using the system call function provided by UNIX OS. A left-to-right HMM has been considered appropriate for processing those signals for which their properties change over time. The underlying state sequence associated with the model has the property that as time increases, the state index increases or stays in the same state. That is, the state always proceeds from left to right. The basic model consists of six states that have less than two direct transitions to the right state. Few initial conditions are given to this model, and these initial conditions are equivalent to all transient models. The re-estimation algorithm of the HMM might give a local minimum of the likelihood function. It is important to choose initial estimates of the HMM parameters so that the local minimum is the global minimum. Experience has shown that uniform initial estimates work well.30 We also choose uniform initial estimates in this implementation by assuming that the observation symbol probability is equivalent to each state. The training is performed by a forward algorithm and a backward algorithm, and then by making a re-estimate from the Baum-Welch algorithm in each model, given the multiple input observations.30 The re-estimation is done until the conver¯ ≥ P(O | λ), that is, the new model estimates are more gence condition, P(O | λ) likely to produce the given observation sequence O, is satisfied in each model. The
803
HIDDEN MARKOV MODEL-BASED REAL-TIME
probability, P(O | λ), is calculated by the optimal path, which is obtained by the Viterbi algorithm for the given input observations in each model. The transients are classified by examining which model has the highest probability for the given input observations. The prototype of the transient identification system was implemented using an HP747i industrial workstation, and the programming was done using the C language. 4.
EXPERIMENTAL RESULTS
In this section, experimental test results for a base model are described. Then, the selected model is suggested and the improved model is proposed to improve identification accuracy. The base model consists of a max-min normalization method, a SOM clustered by training and test data, and a six-state HMM identifier. The experimental tests have been carried out after training of the vector quantizer and HMM identifier. Table II shows the results of the base model off-line test. In this experiment, the following four test cases were used to compare the trained or nontrained data for operating modes and break sizes. • • • •
Case I: Trained operating mode, non-trained break size Case II: Non-trained operating mode, trained break size Case III: Non-trained operating mode, non-trained break size Case IV: Trained operating mode, trained break size
As presented in this table, most of the transients are correctly identified when given trained break size and non-trained operating mode. But in the case of nontrained break size, the recognition rate is remarkably lower than the trained break size. There is almost no difference when comparing the case of the trained and the non-trained operating mode. But the identification rate for the non-trained break size is lower than the non-trained operating mode. We estimated that the HMM can absorb these break size differences, but in real implementation, the HMM cannot completely absorb the break size differences. In Case IV, with both trained operating Table II. Off-line test results for base model. Recognition rates, % Transient
Case Case I
Case II
Case III
Case IV
Average Except Case IV
ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Average
— 100 27.3 45.5 — 0.0 72.7 100 100 63.6
100 100 100 100 100 100 80.0 100 100 97.8
— 100 30.0 50.0 — 0.0 70.0 100 100 64.3
100 100 100 100 100 100 100 100 100 100
100 100 51.6 64.5 100 32.3 74.2 100 100 76.8
804
KWON, KIM, AND SEONG 100
80 Identification rate (%)
ATWS FWLB LOCA
60
LSLC MSIV MSLI MSLO
40
PORV SGTR Avg.
20
0
0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 Time (seconds)
Figure 5. On-line identification rate for base model.
mode and trained break size, the identification rate is 100 percent. This means the trained data can be completely identified. The on-line test for the base model was performed using on-line test data that covers various operating modes and break sizes for each transient. The on-line test results are depicted in Figure 5. The total identification rate of the on-line test reaches 95.8 percent within 79 seconds, but the immediate detecting rate is only 44.3 percent. In the case of LOCA, MSLI, or PORV, the transients are correctly identified from incorrect transients. In the initial stage of these transients, there are no distinctive features between preceding patterns of identified and misidentified transients. The distinctive features appeared after a few seconds; then, the HMM identifier was able to identify the correct transient types. It takes some time to detect an SGTR transient, because its distinctive features only appear after some time in a real situation. The normalization methods are compared based on the assumption that the input normalization might influence the result of classification. The identification rate of each transient at different normalization methods is shown in Figure 6. As indicated in Figure 1, the normalization results of the mean-C1 method are not proper; the normalized data are concentrated near the mean. As a consequence of the normalization results of the mean-C1 method, the identification rate is lower than in the max-min or mean-C2 methods. The identification rate of max-min is higher than the other two methods. The max-min method has advantages in the present case. Consequently, the max-min method is chosen for the selected model. To choose the proper clustering algorithm, two types of clustering methods are compared, and the results are depicted in Figure 7. Considering the results, the K-means algorithm has better performance than the SOM. Finally, the K-means algorithm method with training data is chosen to implement the selected model.
805
HIDDEN MARKOV MODEL-BASED REAL-TIME
100 ATWS
Identification rate (%)
80
FWLB LOCA LSLC
60
MSIV MSLI MSLO
40
PORV SGTR
20
Avg.
0 max-min
mean-C1 Normalization methods
mean-C2
Figure 6. Identification rate based on normalization methods.
Identification rate (%)
100 ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Avg.
80 60 40 20 0 SOM
K-means Clustering methods
Figure 7. Identification rate based on clustering methods.
The selected model is suggested according to the results of the previous experimental tests. The selected model means adopting a proper normalization method, clustering algorithm, and number of HMM states. According to the results of previous experimental tests, the selected model has a max-min normalization method, K-means clustering algorithm, and six states of left-to-right HMM. The total identification rate of the selected model is 100 percent within 17 seconds, and its immediate detection rate is 54.0 percent. The total identification and immediate detection rate of the selected model is higher than the base model. In the selected model, the detection rate of correct transients from incorrect transients is significantly reduced from the base model. According to the results of the above experimental tests, it can
806
KWON, KIM, AND SEONG 100 90
Identification rate (%)
80
ATWS FWLB LOCA LSLC MSIV MSLI MSLO PORV SGTR Avg.
70 60 50 40 30 20 10 0 0
2
4
6
8 10 12 Random noise (%)
14
16
18
20
Figure 8. Test results of superimposing random noise.
be concluded that the selected model has good performance, but it cannot be said that the selected model is robust. Therefore, three more experimental tests were performed, including superimposing random noise, adding systematic error, and testing by untrained transients to verify the robustness of the selected model. The random noise is superimposed on the original test data, except for two variables, such as position of the main steam isolation valve and reactor trip signal. The terminology “2 percent of random noise” means a maximum ±2 percent of random noise is superimposed on test data in the normalizing progress. The test results of superimposing random noise are depicted in Figure 8. In the case of 2 percent to 8 percent of random noise, the identification rates are the same as the no random noise case. The identification rate decreases slightly at 10 percent of random noise. But, the identification rate abruptly decreases for more than 12 percent of random noise. It can be said that the proposed transient identification system is robust within 10 percent of random noise. In neural network applications, the recognition rate linearly decreases by increasing the noise level,2 or the recognition rate is maintained to 16 percent of the noise level, then the rate abruptly decreases.10 Additional tests were performed in which the systematic error signal is added differently from the random noise. The first error signal is a “loss of pressurizerlevel signal,” which has large variations when the transients occur. In this case, the identification rate is significantly lower than with no error signal. From this test result, to ensure the performance of the transient identification system, valid signals through a signal validation process should be provided. In the application of a neural network case, the system can identify the transient properly even though some sensor signals are missed.36 But the second error signal is a “loss of main steam flow signal,” which has small variations when the transients occur. In this case, the identification
807
HIDDEN MARKOV MODEL-BASED REAL-TIME
100 ATWS
Identification rate (%)
80
FWLB LOCA LSLC
60
MSIV MSLI
40
MSLO PORV SGTR
20
Avg.
0 No error
Loss of pressurizerlevel signal
Loss of main steam flow signal
Systematic error
Figure 9. Test results of adding systematic error.
rate is the same as if there was no error signal. This means the error signals that have small variations have almost no effect on identification rate. The test results of adding a systematic error are depicted in Figure 9. It is desirable that the never-trained transient should be classified as an unknown transient. In particular, a severe accident can happen because of an inadequate operation due to incorrect identification. The implemented system can classify the unknown transient similar to the other neural network applications.3,9,16 A transient is classified as unknown if the output path probability of the HMM identifier is less than the threshold that has the least output path probability of all test data. The test results of untrained transients are summarized in Table III. As shown in Table III, three untrained transients are classified as unknown transients. But four untrained transients are identified as normal states. In the case of the incorrect classification, the HMM identifier might not be exactly identified, because the clustering results are very similar to the misclassified transient. In the selected model, there still exist temporary misclassified transients. Finally, two heuristic training approaches are attempted to improve the classification Table III. Test results of untrained transient. Untrained Transient Description Drop of all control rods in control bank group A Reactor coolant pump trip Charging line rupture Turbine control valve failure (close) Steam line rupture (100 cm2 ) Failure in feedwater controller (loss of steam flow signal) Failure in pressurizer controller: maximum failure
Reactor Power (%)
Test Result
95 88 78 90 100 100 95
Unknown Unknown Normal state Unknown Normal state Normal state Normal state
808
KWON, KIM, AND SEONG 100
Identification rate (%)
90
80 Base model
70
Selected model Improved model
60
50
40
0
20
40
60
80
Time (seconds) Figure 10. On-line identification rate for three models.
accuracy. One of these approaches is corrective training.37 The heuristic corrective training method re-estimates model parameters using a validation data set that is not included in the training or test data sets. After the re-estimation of HMM parameters using the training data set, transient identification is performed on the validation data. If any transient is misclassified, the estimated model parameters are adjusted to reduce the probability of misclassified transients. In real implementation, the corrective training is applied to only ATWS and FWLB transients to improve the immediate detection rate. The other heuristic approach is the principal component method. The principal component is applied to increase discriminating power between two transients that are expressed by similar patterns, such as FWLB and LOCA transients. This method adds a weighting factor to important variables that have large impacts on identification results when vector quantization is progressed. The secondary radiation variable is chosen as an important variable to differentiate FWLB and LOCA transients in this implementation. Consequently, the classification rate is slightly increased when the principal component method is applied. In conclusion, Figure 10 shows the classification rate of three on-line tests over time. The classification rate of the selected model is much better than the base model, and the classification rate of the improved model is a little better than the selected model. 5.
CONCLUSIONS
The proposed transient identification system has many advantages, as described above. However, there are still some problems that should be solved before it can actually be applied to operating NPPs. Further efforts are being made to improve the
HIDDEN MARKOV MODEL-BASED REAL-TIME
809
system performance and robustness to demonstrate reliability and accuracy to the required level. Following are the identified problems and suggestions learned from this work. It is a major problem that the training data are extremely rare compared with other applications. The HMM-based identifier is more difficult to train properly because it tends to require more training data. For this reason, research on a modified HMM structure or a new algorithm for the estimation of HMM parameters is suggested. As shown in experimental tests, the identification performance heavily depends on clustering methods. Therefore it is very important to find more efficient clustering methods that are suitable for the proposed system. The HMM-based identifier cannot completely resolve the “don’t know” issue. Future work should solve this problem. A fitness measure also needs to be developed to confirm the degree of belief. Besides the improvements described in the improved model, it needs to be integrated with other features such as knowledge processing38 or neural networks 22 to improve its accuracy. Artificial intelligence (AI) techniques such as neural networks, fuzzy logic, and HMM that are applicable to NPPs require careful consideration.39 In particular, AIbased, safety-critical systems should be treated with caution and need confirmation from the regulatory body. Acknowledgment The authors wish to thank the anonymous reviewers for their useful comments and corrections. This work has been carried out under the Nuclear Research and Development Program by the Ministry of Science and Technology in Korea.
References 1. 2. 3.
4. 5. 6. 7. 8. 9.
Kim IS. Computerized systems for on-line management of failures: A state-of-the-art discussion of alarm and diagnostic systems applied in the nuclear industry. Reliability Eng Syst Safety 1994;44:279–295. Ikonomopoulos A, Uhrig RE, Tsoukalas L. A hybrid neural network-fuzzy logic approach to nuclear power plant transient identification. AI91 Frontiers in Innovative Computing for the Nuclear Industry. Jackson, WY; Sep 15–18, 1991. pp 217–226. Bartal Y, Lin J, Uhrig R. Transients identification in nuclear power plants using probabilistic neural networks and the problem of knowledge extrapolation. In: 9th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 24–26, 1995. pp 49.01– 49.08. U.S. Nuclear Regulatory Commission. Functional criteria for emergency response facilities, NUREG-0696; Feb 1981. Bruckner LA. On Chernoff faces. In: Wang PCC, editor. Graphical representation of multivariate data. New York: Academic Press; 1978. U.S. Nuclear Regulatory Commission. Human factors acceptance criteria for the safety parameter display system, NUREG-0835; Oct 1981. Guo Z, Uhrig RE. Accident scenario diagnostics with neural networks. In: 8th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 1992. pp 53.01–53.11. Basu A. Nuclear power plant status diagnostics using a neural network with dynamic node architecture. MS thesis, Iowa State University; 1992. Bartal Y, Lin J, Uhrig R. Nuclear power plant transient diagnostics using artificial neural networks that allow “don’t-know” classifications. Nucl Technol 1995;110:436–449.
810 10. 11. 12. 13. 14. 15.
16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33.
KWON, KIM, AND SEONG Jeong E, Furuta K, Kondo S. Identification of transient in nuclear power plant using neural network with implicit time measure. In: Proc Topical Meeting on Computer-Based Human Support Systems. Philadelphia, PA; June 25–29, 1995. pp 467–474. Parlos AG, Atiya AF, Chong KT, Tsai WK. Nonlinear identification of process dynamics using neural networks. Nucl Technol 1992;97:79–96. Roh CH, Chang HS, Kim HG, Chang SH. Identification of reactor vessel failures using spatiotemporal neural networks. IEEE Trans Nucl Sci 1996;43(6):3223–3229. Iijima T. Application study of fuzzy logic method for plant-state identification. HWR-432, OECD Halden Reactor Project; Dec 1995. Lin J, Bartal Y, Uhrig R. Using similarity based formulas and genetic algorithms to predict the severity of nuclear power plant transients. In: Proc 9th Power Plant Dynamics, Control & Testing Symposium. Knoxville, TN; May 24–26, 1995. pp 53.01–53.09. Jeong E, Furuta K, Kondo S. Identification of transient in nuclear power plant using adaptive template matching with neural network. In: Nuclear Plant Instrumentation, Control, and Human-Machine Interface Technologies. Pennsylvania State University; May 6–9, 1996. pp 243–250. Hines JW, Miller DW, Hajek BK. A hybrid approach for detecting and isolating faults in nuclear power plant interacting systems. Nucl Technol 1996;115:342–358. Zhu Q. Hidden Markov model for dynamic obstacle avoidance of mobile robot navigation. IEEE Trans Robot Automat 1991;7(3):390–397. DeWitt MR. High range resolution radar target identification using the PRONY model and hidden Markov models. MS thesis, Air Force Institute of Technology, WPAFB OH; Dec 1992. Yamato J, Ohya J, Ishii K. Recognizing human action in time-sequential images using hidden Markov model. In: Proc Computer Vision and Pattern Recognition. Urbana-Champaign, IL; 1992. pp 379–385. Starner TE. Visual recognition of American sign language using hidden Markov models. MS thesis, MIT; Jan 1995. Thoraval L, Carrault G, Bellanger J. Heart signal recognition by hidden Markov models: The ECG case. Methods Info Med 1994;33(1):10–14. Kundu A, Chen G. An integrated hybrid neural network and hidden Markov model classifier for sonar signal classification. ICASSP 1995;3587–3590. Brand M, Oliver N, Pentland A. Coupled hidden Markov models for complex action recognition, CVPR97. San Juan, Puerto Rico; June 17–19, 1997. pp 994–999. Hatzipantelis E, Penman J. The use of hidden Markov models for condition monitoring electrical machines. In: 6th Int Conf on Electrical Machines and Drives. Sept 8–10, 1993. Smyth P. Hidden Markov models for fault detection in dynamic systems. Pattern Recognit 1994;27(1):149–164. Fielding KH, Ruck DW. Recognition of moving light displays using hidden Markov models. Pattern Recognit 1995;28(9):1415–1421. Couvreur C, Fontaine V, Gaunard P, Mubikangiey G. Automatic classification of environmental noise events by hidden Markov models. Appl Acoustics 1998;54(3):187–206. Kulp D, Haussler D, Reese MG, Eeckman FH. A generalized hidden Markov for the recognition of human genes in DNA. In: Proc 4th Int Conf on Intelligent Systems for Molecular Biology. Washington University, St. Louis; June 12–15, 1996. pp 134–142. Smyth P. Tutorial material; machine learning: Theory and application. In: Proc 3rd World Congress on Expert Systems, Seoul. Korea; Feb 5–9, 1996. Huang XD, Ariki Y, Jack MA. Hidden Markov models for speech recognition. Edinburgh: Edinburgh University Press; 1990. Couvreur C. Environmental sound recognition: A statistical approach. Ph.D. thesis, Facult´e Polytechnique de Mons, Belgium; June 1997. Rabiner LR. A tutorial on hidden Markov models and selected application in speech recognition. Proc IEEE 1989;77(2):257–285. Kozma R, Yokoyama Y, Kitamura M. Intelligent monitoring of NPP anomalies by adaptive neuro-fuzzy signal processing system. In: Proc Topical Meeting on Computer-Based Human Support Systems. Philadelphia, PA; June 25–29, 1995. pp 449–456.
HIDDEN MARKOV MODEL-BASED REAL-TIME
811
34. Kohonen T. The self-organizing map. Proc IEEE 1990;78(9):1464–1480. 35. Kwon KC, Song SJ, Park WM, Lyu SP. The real-time functional test facility for advanced instrumentation and control in nuclear power plants. IEEE Trans Nucl Sci 1999;46(2). 36. Cheon SW, Chang SH. Application of neural networks to a connectionist expert system for transient identification in nuclear power plants. Nucl Technol 1993;102:177–191. 37. Applebaum T, Hanson B. Enhancing the discrimination of speaker independent hidden Markov models with corrective training. In: Proc ICASSP-89. Glasgow, Scotland; May 1989. pp 302–305. 38. Ohga Y, Seki H. Abnormal event identification in nuclear power plants using a neural network and knowledge processing. Nucl Technol 1993;101:159–167. 39. Wainwright N. A regulator’s viewpoint on the use of AI in the nuclear industry. Nucl Energy 1995;34(2):93–97.
