Batch-to-batch control of fed-batch processes ... - Semantic Scholar
Neural Comput & Applic (2008) 17:425–432 DOI 10.1007/s00521-007-0142-6
ORIGINAL ARTICLE
Batch-to-batch control of fed-batch processes using control-affine feedforward neural network Zhihua Xiong Æ Yixin Xu Æ Jie Zhang Æ Jin Dong
Received: 6 February 2007 / Accepted: 2 August 2007 / Published online: 8 September 2007 Springer-Verlag London Limited 2007
Abstract A control strategy for fed-batch processes is proposed based on control affine feed-forward neural network (CAFNN). Many fed-batch processes can be considered as a class of control affine nonlinear systems. CAFNN is constructed by a special structure to fit the control affine system. It is similar to a multi-layer feedforward neural network, but it has its own particular feature to model the fed-batch process. CAFNN can be trained by a modified Levenberg–Marquardt (LM) algorithm. However, due to model-plant mismatches and unknown disturbances, the optimal control policy calculated based on the CAFNN model may not be optimal when applied to the fed-batch process. In terms of the repetitive nature of fed-batch processes, iterative learning control (ILC) can be used to improve the process performance from batch to batch. Due to the special structure of CAFNN, the gradient information of CAFNN can be computed analytically and applied to the batch-to-batch ILC. Under the ILC strategy from batch to batch, endpoint product qualities of fed-batch processes can be improved gradually. The proposed control
Z. Xiong (&) Y. Xu Department of Automation, Tsinghua University, Beijing 100084, People’s Republic of China e-mail: [email protected] J. Zhang School of Chemical Engineering and Advanced Materials, University of Newcastle, NE1 7RU Newcastle upon Tyne, UK e-mail: [email protected] J. Dong Supply Chain Management and Logistics, IBM China Research Lab, Beijing 100094, People’s Republic of China e-mail: [email protected]
scheme is illustrated on a simulated fed-batch ethanol fermentation process. Keywords Control affine system Neural network Iterative learning control Fed-batch process
1 Introduction Fed-batch processes are suitable for responsive manufacturing, and the general interests are concerned with determining the feeding policy to the reactor that will give the maximum amount of the desired product [1]. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control problem, considerable difficulties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because, in general, fed-batch processes are nonlinear dynamic systems [2, 3]. In fed-batch processes, it is very important for optimal control to obtain an accurate model that can provide accurate long range predictions [1]. To overcome the difficulties in building mechanistic models, empirical models based on process input–output data can be utilized. Neural network (NN) has been proven to be able to approximate any continuous nonlinear functions and has been applied to nonlinear process modeling and control [4]. If properly trained and validated, NN models can be used to predict steady-state and dynamic process behavior reasonably well, hence, leading to improved process optimization and control performance [5, 6]. Many fed-batch processes can be considered as a class of control affine nonlinear systems. In our previous work, control affine feed-forward neural network (CAFNN) is proposed to model the fed-batch process [7]. CAFNN is similar to a multi-layer feed-forward neural network but
123
426
has its own special structure. The network can be trained by a modified Levenberg–Marquardt (LM) algorithm, and it offers an effective and simple optimal control strategy, in which the gradient information can be computed analytically in a recursive fashion. However, the neural network model developed is only an approximation of the fed-batch process and its prediction offsets may occur due to modelplant mismatches and unknown disturbances [7]. Then optimal control policy calculated from the CAFNN model may not be optimal when applied to the actual process. Several methods have been proposed to address this issue. Xiong and Zhang [6] propose a re-optimization scheme for batch processes to overcome the detrimental effects of model-plant mismatches and unknown disturbances. Zhang [8] presents a reliable optimization method for batch process based on bootstrap aggregated neural network models by incorporating confidence bounds of those model prediction into the optimization objective function. The repetitive nature of fed-batch processes allows the information of previous batch runs to be used to improve the performance of next batch runs in the fashion of iterative learning control (ILC) [9–11]. The basic idea of ILC is to update the control trajectory for a new batch run using the information from previous batch runs so that the output trajectory converges asymptotically to the desired reference trajectory. Refinement of control signals based on ILC can significantly enhance the performance of tracking control systems. Optimal ILC is one of important methods for designing an iterative learning law, in which a quadratic objective function is to be optimized [12]. Lee et al. in several related articles [10, 13, 14] proposed the quadratic criterionbased ILC (Q-ILC) approach for tracking control for temperature of batch processes based on a linear time varying (LTV) tracking error transition model. In our previous work [11], an ILC strategy for the tracking control of product quality in batch processes based on an LTV perturbation model was proposed. The convergence of ILC law based on the batch-wise LTV perturbation model was also demonstrated. To address the problem of model-plant mismatches, the model prediction errors in the previous batch run are added directly to the model predictions for the current batch run. Since ILC is well developed for linear models, most of the ILC-based batch-to-batch control schemes are based on some kinds of linear models (e.g., [9, 12, 13]). Nonlinear model based ILC schemes for batch and fed-batch processes have also recently been proposed [6, 15]. Xiong and Zhang [6] present a recurrent neural network based ILC scheme for batch processes where the prediction errors of recurrent neural network are filtered from previous batches and added to the model predictions for the current batch, and then optimization is performed based on the updated predictions. Zhang [15] presents an ILC scheme based on a feed-forward neural network model where the network is linearized
123
Neural Comput & Applic (2008) 17:425–432
around the current batch. Based on the linearized model, the control policy for next batch is updated using ILC to minimize the tracking errors at the end of the next batch. In this study, CAFNN-based ILC strategy is proposed to improve the tracking control performance for fed-batch processes. To improve the accuracy of model predictions, the model errors of the immediately previous batch are also used to modify the CAFNN model prediction. Because the gradient information can be calculated directly from its special structure, the linearized model of CAFNN is easy to be obtained analytically and then applied to the batch-to-batch ILC. Therefore, based on the batch-to-batch ILC law, the control policy for a next batch is modified from the previous control profile. This procedure is repeated from batch to batch, and then the performance of the tracking control of the endpoint product qualities can be improved gradually. The rest of this paper is structured as follows: the structure of CAFNN is presented in Sect. 2. In Sect. 3, the batch-to-batch ILC strategy for fed-batch processes based on a CAFNN model is described in detail, where the algorithm for the recursive computation of the gradient information is also presented. The proposed method is demonstrated on a simulated fed-batch ethanol fermentation process in Sect. 4. Finally Sect. 5 draws some concluding remarks.
2 Control-affine feedforward neural network The control affine feed-forward neural network (CAFNN) model [7] is reviewed in this section. In many biochemical processes, the reactors are operated in fed-batch mode, where the feed rate is used for control, as shown in Fig. 1. In such a fed-batch process, the general interests are concerned with determining the
Fig. 1 A diagram of fed-batch process
Neural Comput & Applic (2008) 17:425–432
427
feeding policy to the reactor that will give the maximum amount of the desired product. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control problem, considerable difficulties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because fed-batch processes are frequently nonlinear and state constrained [16, 17]. In this study, we consider that fed-batch processes usually have a fixed batch length (tf) and all batches run from the same initial condition. Fed-batch process usually can be considered as a class of control affine nonlinear systems, mathematically formulated as [7]: x_ k ¼ f ðxk Þ þ gðxk Þuk ;
xk ð0Þ ¼ xk0
ð1Þ
yk ¼ xk
ð2Þ
where the subscript k denotes the batch index, x [ Rp a vector of process states, u [ Rq a vector of process inputs, y [ Rp a vector of process output, respectively. It is assumed here that the duration of batch run consists of N sampling intervals (i.e., N = tf/h). From above control affine system (1) and (2) and considering its special characteristics, a discrete time model for the system can be described as yk ðt þ 1Þ ¼ f ðyk ðtÞ; . . .; yk ðt n þ 1ÞÞ þ gðyk ðtÞ; . . .; yk ðt n þ 1ÞÞuk ðtÞ
ð3Þ
where n (n p) is the order of the nonlinear system. To model this kind of nonlinear system, a special neural network, CAFNN, is proposed [7] and its topology is shown in Fig. 2. The dashed frame in Fig. 2 is consisted of an inner feed-forward neural network with a hidden layer, thus the CAFNN model can be considered similarly as a two-hidden-layer network. Model output of CAFNN is formulated as ^ k ðtÞÞ y^k ðt þ 1Þ ¼ W 0 ðtÞT Fðv ¼ f^ðvk ðtÞÞ þ g^ðvk ðtÞÞuk ðtÞ
ð4Þ
where vk ðtÞ ¼ ½yk ðtÞ; yk ðt þ 1Þ; . . .; yk ðt n þ 1ÞT ; W0(t) = ^ k ðtÞÞ ¼ ½f^ðvk ðtÞÞ; g^ðvk ðtÞÞT ; and f^ and g^ are [1, uk(t)]T, Fðv predictions of the inner feed-forward network, respectively. ^ k ðtÞÞ can be calculated further according to the Fðv topology of feed-forward network ^ k ðtÞÞ ¼ Fðv
nh X m¼1
w2jm a
nI X
!! w1ml vk ðlÞ
þ
l1m
þ l2j
Fig. 2 The structure of control affine feed-forward neural network
model, and a(z) = 1/(1 + ez), respectively. For a CAFNN with nI inputs and nh hidden neurons, its structure can be symbolized as CAFNN: nI-nh-21. There are some differences between a CAFNN and a multi-layer feed-forward neural network (MFNN) with two-hidden-layers and two hidden neurons on the second hidden layer. To model a fed-batch process, the input vector of such an MFNN is v(t) = [y(t),…,y(tn + 1), u(t)]T, which is symbolized as MFNN: (nI + 1)-nh-21, but the input vector of CAFNN only includes values of output, i.e., v(t) = [y(t),…,y(tn + 1)]T. A CAFNN can be trained using a number of training methods, such as the back-propagation method, the conjugate gradient method, Levenberg–Marquardt (LM) optimization, or methods based on genetic algorithms. In this paper, CAFNN is trained by using a modified LM algorithm with regularization presented in our previous work [7]. After the CAFNN is trained, the network can predict recursively the endpoint output y^k ðNÞ for the kth batch given the initial conditions (u0, y0) and the whole control profile. The model prediction error ek(N) for the kth batch is then calculated by ek ðNÞ ¼ yk ðNÞ y^k ðNÞ
ð6Þ
where yk(N) is the actual product quality values at the end of a batch. To improve the accuracy of model predictions, the model errors of the immediately previous batch are used to modify the CAFNN model prediction: y~k ðNÞ ¼ y^k ðNÞ þ ek1 ðNÞ
ð7Þ
where y~k ðNÞ is the modified prediction of endpoint product quality values.
ð5Þ
l¼1
where j = 1, 2, w1ml and w2jm are connection weights, l1m and l2j are the thresholds, nI and nh are the numbers of nodes in the input layer and hidden layer of the inner NN
3 Batch-to-batch control of fed-batch processes In fed-batch processes, the maximum amount of the desired product is usually considered by determining the optimal
123
428
Neural Comput & Applic (2008) 17:425–432
feed rate to the reactor. Based on the CAFNN model developed above, the optimal control policy U can be obtained by solving the following optimization problem [7] min /ð yðNÞÞ U
ð8Þ
where / is a scalar objective function in terms of the endpoint product qualities. Due to the CAFNN model, above optimization problem is usually solved using a nonlinear programming method such as the sequential quadratic programming (SQP) method [7], iterative dynamic programming (IDP) method [16, 17], and so on. As the CAFNN only approximates the fed-batch processes, prediction offsets are unavoidable due to model-plant mismatches. Then optimal control policy calculated from the CAFNN model may not be optimal when applied to the actual process [6, 7]. However, the repetitive nature of fed-batch processes allows the information of previous batch runs to be used in order to improve the performance of next batch runs [10, 15]. In this study, batch-to-batch ILC strategy is proposed to improve the control performance and limit the deterioration. Let us define the control and product quality sequences through a whole batch duration as Uk ¼ ½uTk ð0Þ; uTk ð1Þ; . . .; uTk ðN 1ÞT Yk ¼ ½yTk ð1Þ; yTk ð2Þ; . . .; yTk ðNÞT :
ð9Þ ð10Þ
The aim of batch-to-batch control is to use model-based ILC to find an updating mechanism for the control sequence of the feed rate Uk+1 of a new batch so that the measured endpoint product qualities yk+1(N) converges asymptotically towards the desired values yd(N). Using the modified prediction errors upon completion of the kth batch to update the control policy for the (k + 1)th batch, the following quadratic objective function is considered [9, 15] Jkþ1 ¼
i 1h ekþ1 ðNÞ þ DUTkþ1 RDUkþ1 e~kþ1 ðNÞT Q~ 2
ð11Þ
where e~kþ1 ðNÞ is the tracking error of the endpoint value e~kþ1 ðNÞ ¼ yd ðNÞ y~kþ1 ðNÞ
ð13Þ
and Q and R are weighting matrices with appropriate dimensions, respectively.
123
N 1 X oyðNÞ y^kþ1 ðNÞ ¼ y^k ðNÞ þ ouðiÞ i¼0
ðukþ1 ðiÞ uk ðiÞÞ þ dkþ1
uk ðiÞ
y^k ðNÞ þ GTk DUkþ1
ð14Þ
where Gk ¼
oyk ðNÞ oyk ðNÞ oyk ðNÞ ; ; . . .; ouk ð0Þ ouk ð1Þ ouk ðN 1Þ
T ð15Þ
and dk+1 is the modeling error due to the linearization. It is reasonable to assume here that dk+1 can be omitted by neglecting the higher-order terms. According to the special structure of CAFNN in (5), the gradient information rJ(u(k)) can be computed directly. Let us define oyk ðNÞ of^ðyk ðNÞÞ o^ gðyk ðNÞÞ ¼ þ uk ðN 1Þ: oyk ðN lÞ oyk ðN lÞ oyk ðN lÞ
ð16Þ
Then the gradient information qyk/quk can be computed by 8 i¼N1 g^ðyk ðNÞÞ; oyk ðNÞ < P n oyk ðNÞ oyk ðNlÞ ¼ ð17Þ i\N 1 : : oyk ðNlÞ ouk ðiÞ ; ouk ðiÞ l¼0
It should be noted that in above (17) the following partial derivatives are zeros
ð12Þ
DUk+1 is the difference of control sequences between two adjacent batches DUkþ1 ¼ Ukþ1 Uk
In order to limit the deterioration of control performance due to model-plant mismatches and unknown disturbances, a linearized model around the neural network model is obtained. In many batch and fed-batch processes, subtracting the time-varying nominal trajectories from the process operation trajectories removes the majority of the process nonlinearity and allows linear modeling methods to perform well on the resulting perturbation variables [18]. The linearization of nonlinear fed-batch processes around the nominal trajectory is a good approximation to the real processes as the input change is typically small. According to the CAFNN model, the prediction for the (k + 1)th batch can be approximated using the first order Taylor series expansion based on the kth batch:
oyk ðN lÞ ¼ 0; ouk ðiÞ
i N l; ðl ¼ 1; . . .; nÞ:
ð18Þ
By computing qyk(j)/quk(i) recursively from j = i + 1 to j = N, the gradients qyk(N)/quk(i), (i = 0, 1,…, N1) can be obtained. The recursive algorithm for computing the gradient qyk(j)/ quk(i) is outlined as follows [7]
Neural Comput & Applic (2008) 17:425–432
8 g^ðyk ðjÞÞ; oyk ðjÞ < P n oyk ðjÞ oyk ðjlÞ ¼ ouk ðiÞ : oyk ðjlÞ ouk ðiÞ ;
429
i¼j1 i\j 1
ð19Þ
l¼0
ekþ1 ðNÞ ¼ yd ðNÞ y^kþ1 ðNÞ þ ekþ1 ðNÞ ¼ yd ðNÞ y~kþ1 ðNÞ ek ðNÞ ekþ1 ðNÞ ¼ e~kþ1 ðNÞ Dekþ1 ðNÞ:
ð29Þ
oyk ðjÞ of^ðyk ðjÞÞ o^ gðyk ðjÞÞ ¼ þ uk ðj 1Þ oyk ðj lÞ oyk ðj lÞ oyk ðj lÞ
ð20Þ
Then above batch-to-batch control law (26) can be calculated further by
nh of^ðyk ðjÞÞ X ¼ w2 dðv ðjÞÞw1ml oyk ðj lÞ m¼1 1m k
ð21Þ
Ukþ1 ¼ Uk þ Kk ek ðNÞ:
ð30Þ
Substituting (29) to (28), we have
nh X
o^ gðyk ðjÞÞ ¼ w2 dðv ðjÞÞw1ml oyk ðj lÞ m¼1 2m k
ð22Þ
ekþ1 ðNÞ ¼ I GTk Kk ek ðNÞ Dekþ1 ðNÞ:
dðvk ðjÞÞ ¼ Oðvk ðjÞÞð1 Oðvk ðjÞÞÞ
ð23Þ
Theorem 1 Let {ek+1(N)} be the tracking error sequence under the above optimal ILC law (30). If the following conditions are satisfied, V k = 1, 2,...,?
ð24Þ
I GT Kk k\1; k
Oðvk ðjÞÞ ¼ a
nI X
! w1ml yk ðj lÞ þ l1m
l¼1
where l = 1,..., n, i = 0, 1,..., N1, j = 1,..., N. In above equations, yk are predictions of CAFNN. Actually the measured product qualities can also be used to correct the gradient information as a feedback during batch-to-batch control. Substituting (7) and (14) to (12), an iterative relationship for e~kþ1 ðNÞ along the batch index k can be obtained e~kþ1 ðNÞ ¼ yd ðNÞ y^k ðNÞ þ GTk DUkþ1 ek ðNÞ
GTk DUkþ1 ek ðNÞ
ð26Þ
where Kk is the learning rate in the ILC 1 Kk ¼ Gk QGTk þ R Gk Q:
ð27Þ
Substituting (26) to (25), we have e~kþ1 ðNÞ ¼ I GTk Kk ðe~k ðNÞ Dek ðNÞÞ:
then kekþ1 ðNÞk will converge to a small value. Proof Using (31) recursively, the following relationships for ek+1(N) can be obtained ekþ1 ðNÞ ¼ I GTk Kk ek ðNÞ Dekþ1 ðNÞ ¼ I GTk Kk I GTk1 Kk1 ek1 ðNÞ I GTk Kk Dek ðNÞ Dekþ1 ðNÞ
i¼0
ð25Þ
where Dek ðNÞ ¼ ek ðNÞ ek1 ðNÞ is the prediction difference between two adjacent batches. Setting qJ/ qUk+1 = 0 in the objective function (11) and through straightforward computation, the batch-to-batch ILC law can be calculated as DUkþ1 ¼ Kk ðe~k ðNÞ Dek ðNÞÞ
ð32Þ
¼ " # k Y T ¼ I G i K i e0
¼ yd ðNÞ ðy~k ðNÞ ek1 ðNÞÞ
¼ e~k ðNÞ GTk DUkþ1 Dek ðNÞ
kDek ðNÞk E\1
ð31Þ
ð28Þ
The tracking error of the endpoint value is defined as ek+1(N) = yd(N) yk+1(N). From above definitions of ek(N) and Dek(N), it holds
" # ! k k
Y X T I Gj Kj Dei ðNÞ Dekþ1 ðNÞ i¼1
j¼i
ð33Þ where e0 is the initial tracking error and is assumed to be obtained, G0 is the initial model obtained from the CAFNN, and K0 is set to 0, which represents that the initial learning rate in ILC law is not calculated, respectively. It is derived from (33) that " # Y k I GTi Ki e0 kekþ1 ðNÞk i¼0 " # ! X k k
Y T þ I Gj Kj Dei ðNÞ i¼1 j¼i þ kDekþ1 ðNÞk: ð34Þ Considering I GTi Ki k\1; it holds as follows
123
430
Neural Comput & Applic (2008) 17:425–432
" # Y k T I Gi Ki e0 i¼0
k Y I GT Ki ke0 k ke0 kkk : i
ð35Þ
i¼0
Furthermore, considering kDei ðNÞk E; it also holds " # ! X k k
Y T I Gj Kj Dei ðNÞ þ kDekþ1 ðNÞk i¼1 j¼i " # k k Y X I GTj Kj kDei ðNÞk þ kDekþ1 ðNÞk i¼1
j¼i
k k Y X E I GTj Kj i¼1
" E
j¼i
k k X Y i¼1
! k
!
#
V low ½1; 1; . . .; 1Ukþ1 V hi :
ð40Þ
4 Application to a simulated ethanol fermentation process
þ1
j¼i
ð36Þ
Substituting (35) and (36) to (34), we have kekþ1 ðNÞk ke0 kkk þ
Eð1 kkþ1 Þ : 1k
ð37Þ
Then the following convergence of ek+1(N) can be obtained lim kekþ1 ðNÞk ¼
E ¼ g: 1k
ð38Þ
It means that the tracking error of endpoint value ek+1(N) converges to a small value with respect to the batch index. h Remark According to (27), Kk is related to parameters Q and R. These two weighting matrices can be set specifically so that the condition I GTk Kk k\1 will be satisfied for all batch runs, although the converge rate probably degrades. When the CAFNN model is properly trained and validated, the modeling error can be bounded by a small value and then D ek(N) is also bounded. Therefore, the conditions in (32) can be always satisfied. Based on the above ILC law (30), the control profile Uk+1 for the next batch is modified from the previous control profile Uk. This procedure is repeated from batch to batch, and then the performance of the tracking control of the endpoint product qualities can be improved gradually. In practice, in order to ensure safe and smooth operations of industrial processes, input (e.g., feed rate) and output variables (e.g., product qualities) of fed-batch process are always constrained [1, 6, 10]. For example, if there is no outflow of batch reactor, then the feed rate must be chosen so that the reactor volume does not exceed the
123
ð39Þ
þE
Eð1 k Þ : 1k
k!1
Ulow Ukþ1 Uhi
Above constraints may become active, which usually prevents the convergence to zero tracking error during the batch-to-batch control.
kþ1
¼
physical volume of the reactor. The feed rate may also be constrained to ensure smooth operations. These input and output constraints should be satisfied if they exist. For simplicity of this study on the fed-batch processes, constraints on the magnitude values of control variable and their sum value are only considered as follows
Consider an ethanol fermentation process as shown in Fig. 1, whose mechanistic model in the form of differential algebraic equations (DAE) is described as follows [19]: dx1 x1 ¼ Cx1 u dt x4
ð41Þ
dx2 ð150 x2 Þ ¼ 10Cx1 þ u x4 dt
ð42Þ
dx3 x3 ¼ Dx1 u dt x4
ð43Þ
dx4 ¼u dt
ð44Þ
with C¼
0:408x2 ð1 þ x3 =16Þð0:22 þ x2 Þ
ð45Þ
D¼
x2 ð1 þ x3 =71:5Þð0:44 þ x2 Þ
ð46Þ
where x1 is the cell mass concentration, x2 is the substrate concentration, x3 is the product concentration, and x4 is the liquid volume of the reactor. x4 is limited by the 200 l vessel size. The initial condition is specified as x0 = [1, 150, 0, 10]T. The batch length tf is fixed to be 63.00 hours and divided into N = 10 equal stages (i.e., sampling time h = tf/N = 6.3). The feed rate into the reactor u is used for control and constrained by 0 u 12 (l/h). There is no outflow, so the feed rate must be chosen so that the batch volume does not exceed the physical volume of the reactor. Since x4 is obtained by integrating u, the constraint on u should also be satisfied
Neural Comput & Applic (2008) 17:425–432
431 12 1st batch 5th batch 15th batch
10 8 Feed rate ( L/h )
with [1,1,...,1] Uk 190. The performance index is to maximize the amount of the desired production at the end of batch. The fermentation process was formulated and used for optimal control studies by Hong [19]. This typical biochemical system exhibiting singular control was also studied for global optimization by Hartig [16] and by Bojkov [17]. In this study, the above mechanistic model (41)–(46) is assumed to be not available. Thus a CAFNN based model has to be utilized to model the nonlinear relationship between y = x3 and u, and the CAFNN model is described as
6 4 2 0
yk ðt þ 1Þ ¼ f^ðyk ðtÞ; yk ðt 1Þ; yk ðt 2ÞÞ þ g^ðyk ðtÞ; yk ðt 1Þ; yk ðt 2ÞÞuk ðtÞ:
0
10
20
ð47Þ
For CAFNN modeling, 52 batches of the process operation under different feeding policies were simulated from the mechanistic model to generate the data sets. Then the data sets are divided into training data of 30 batches, testing data of 20 batches, and unseen validation data of 2 batches. After cross-validation, the appropriate topology of CAFNN was selected with 3 input neurons and 10 hidden neurons, i.e., 3-10-2-1. After the CAFNM is trained, the long-range predictions from this CAFNN model on the unseen batch data are shown in Fig. 3. It is clear that the CAFNN model has captured the dynamic trends of the product quality and can be considered as being accurate enough to model the fedbatch fermentation process. For batch-to-batch control, the parameters of ILC were set as Q = I and R = 0.1 IN, and the desired output yd(N) is set to 103.53 [7]. To investigate the performance of the proposed control strategy, the optimal value of the final product concentration is first obtained by solving the
30 time ( h )
40
50
60
Fig. 4 Trajectories of feed rate Uk under the batch-to-batch control
optimization problem based on the CAFNN model, in which the sequential quadratic programming (SQP) method is used to solve the nonlinear optimization problems. Due to CAFNN model-plant mismatches, the value drops to 95.05, an 8.2% reduction. To carry out batch-to-batch control scheme, the calculated control profile only based on the CAFNN is used as the control policy of the first run of the ILC method, as shown in Fig. 4. After 15 batch runs, the actual product concentration is improved to 101.42, only 2.1% reduction, as shown in Fig. 5. Figure 5 shows the root-mean-square-error (RMSE) of tracking error of product quality ek(N) at the batch endpoint under the proposed batch-to-batch control strategy. It can be seen that ek(N) is improved gradually and converges after almost 6 batch runs, which means that the endpoint 9
ymdl(N) = 103.7319 ymCAFNN(N) = 98.7852
8
110
7
90
6
80
3
Product Concentration x ( g/L )
100
70
5
60
4
50 40
3
30
simulated process CAFNN prediction
20
2
10
1 1
0 0
10
20
30 time ( h )
40
50
60
Fig. 3 Predictions of the CAFNN model on the unseen batch data
3
5
7 9 batch number
11
13
15
Fig. 5 Tracking performance of RMSE of ek(N) under the batch-tobatch control
123
432
Neural Comput & Applic (2008) 17:425–432 Acknowledgments The work is partially supported by National Natural Science Foundation of China under grant 60404012, Proj 863 (No. 2007AA04Z193), SRF for ROCS of SEM of China, New Star of Science and Technology of Beijing City, and IBM China Research Lab 2007 UR-Program.
200 180
1st batch 5th batch 15th batch
160 Volume ( L )
140 120
References
100 80 60 40 20 0
0
10
20
30 time ( h )
40
50
60
Fig. 6 Trajectories of reactor volume
value converges asymptotically to the desired one. It should be noted that there are small fluctuation of ek(N) due to the constraints on the feed rate. Figures 4 and 6 show, respectively, the feed rate (control variable) profiles uk and the volume of batch reactor of the 1st, 5th, and 15th batch runs. It can be seen that in Fig. 3 the constraints on the magnitude control values are not violated under the batch-to-batch control. But the constraints on their sum value, i.e., liquid volume of the reactor, are active as shown in Fig. 6. If the sum of control sequence at time i is satisfied with the limit, then the control values after time i must be set to zero, as shown in Fig. 4.
5 Conclusions A batch-to-batch iterative learning control strategy for fedbatch processes is proposed based on control affine feedforward neural network. The CAFNN has its own special structure and is very effective to model fed-batch process. The gradient information can be computed analytically from CAFNN and then applied to the ILC method. By using the information of previous batches, the endpoint product qualities of fed-batch processes are improved gradually from batch to batch. The proposed control scheme is illustrated on a simulated fed-batch ethanol fermentation process. The simulation results show that the proposed method takes effects of the tracking performance. The deterministic model without noises in the fed-batch processes is only studied here. In future works, the fedbatch processes in the presence of model uncertainties will be considered further.
123
1. Bonvin D (1998) Optimal operation of batch reactors—a personal view. J Process Contr 8:355–368 2. Park S, Ramirez WF (1988) Optimal production of secreted protein in fed-batch reactors. AIChE J 34:1550–1558 3. Luus R (1992) On the application of iterative dynamic programming to singular optimal control problems. IEEE Trans Autom Control 37:1802–1806 4. Sjoberg J, Zhang Q, Ljung L, Benveniste A et al (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31:1691–1724 5. Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop PJ (1992) Neural networks for control systems—a survey. Automatica 28:1083– 1112 6. Xiong ZH, Zhang J (2005) A batch-to-batch iterative optimal control strategy based on recurrent neural network models. J Process Contr 15:11–21 7. Xiong ZH, Zhang J (2004) Modelling and optimal control of fedbatch processes using a novel control affine feedforward neural network. Neurocomputing 61:317–337 8. Zhang J (2004) A reliable neural network model based optimal control strategy for a batch polymerisation reactor. Ind Eng Chem Res 43:1030–1038 9. Bristow DA, Tharayil M, Alleyne AG (2006) A survey of iterative learning control. IEEE Control Syst Mag 26:96–114 10. Lee JH, Lee KS, Kim WC (2000) Model-based iterative learning control with a quadratic criterion for time-varying linear systems. Automatica 36:641–657 11. Xiong ZH, Zhang J (2003) Product quality trajectory tracking of batch processes using iterative learning control based on time varying perturbation model. Ind Eng Chem Res 42:6802–6814 12. Amann N, Owens DH, Rogers E (1996) Iterative learning control for discrete-time system with exponential rate of convergence. IEE Proc Control Theory Appl 143:217–224 13. Lee KS, Lee JH (2000) Convergence of constrained model-based predictive control for batch processes. IEEE Trans Autom Control 45:1928–1932 14. Lee KS, Lee JH (2003) Iterative learning control-based batch process control technique for integrated control of end product properties and transient profiles of process variables. J Process Contr 13:607–621 15. Zhang J (2005) A neural network based strategy for the integrated batch-to-batch control and within batch control of batch processes. Trans Inst Meas Control 27:391–410 16. Hartig F, Keil FJ, Luus R (1995) Comparison of optimization methods for a fed-batch reactor. Hung J Ind Chem 23:141–148 17. Bojkov B, Luus R (1996) Optimal control of nonlinear systems with unspecified final times. Chem Eng Sci 51:905–919 18. Russell SA, Kesavan P, Lee JH (1998) Recursive data-based prediction and control of batch product quality. AIChE J 44:2442– 2458 19. Hong J (1986) Optimal substrate feeding policy for fed batch fermentation with substrate and product inhibition kinetics. Biotechnol Bioeng 27:1421–1431
ORIGINAL ARTICLE
Batch-to-batch control of fed-batch processes using control-affine feedforward neural network Zhihua Xiong Æ Yixin Xu Æ Jie Zhang Æ Jin Dong
Received: 6 February 2007 / Accepted: 2 August 2007 / Published online: 8 September 2007 Springer-Verlag London Limited 2007
Abstract A control strategy for fed-batch processes is proposed based on control affine feed-forward neural network (CAFNN). Many fed-batch processes can be considered as a class of control affine nonlinear systems. CAFNN is constructed by a special structure to fit the control affine system. It is similar to a multi-layer feedforward neural network, but it has its own particular feature to model the fed-batch process. CAFNN can be trained by a modified Levenberg–Marquardt (LM) algorithm. However, due to model-plant mismatches and unknown disturbances, the optimal control policy calculated based on the CAFNN model may not be optimal when applied to the fed-batch process. In terms of the repetitive nature of fed-batch processes, iterative learning control (ILC) can be used to improve the process performance from batch to batch. Due to the special structure of CAFNN, the gradient information of CAFNN can be computed analytically and applied to the batch-to-batch ILC. Under the ILC strategy from batch to batch, endpoint product qualities of fed-batch processes can be improved gradually. The proposed control
Z. Xiong (&) Y. Xu Department of Automation, Tsinghua University, Beijing 100084, People’s Republic of China e-mail: [email protected] J. Zhang School of Chemical Engineering and Advanced Materials, University of Newcastle, NE1 7RU Newcastle upon Tyne, UK e-mail: [email protected] J. Dong Supply Chain Management and Logistics, IBM China Research Lab, Beijing 100094, People’s Republic of China e-mail: [email protected]
scheme is illustrated on a simulated fed-batch ethanol fermentation process. Keywords Control affine system Neural network Iterative learning control Fed-batch process
1 Introduction Fed-batch processes are suitable for responsive manufacturing, and the general interests are concerned with determining the feeding policy to the reactor that will give the maximum amount of the desired product [1]. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control problem, considerable difficulties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because, in general, fed-batch processes are nonlinear dynamic systems [2, 3]. In fed-batch processes, it is very important for optimal control to obtain an accurate model that can provide accurate long range predictions [1]. To overcome the difficulties in building mechanistic models, empirical models based on process input–output data can be utilized. Neural network (NN) has been proven to be able to approximate any continuous nonlinear functions and has been applied to nonlinear process modeling and control [4]. If properly trained and validated, NN models can be used to predict steady-state and dynamic process behavior reasonably well, hence, leading to improved process optimization and control performance [5, 6]. Many fed-batch processes can be considered as a class of control affine nonlinear systems. In our previous work, control affine feed-forward neural network (CAFNN) is proposed to model the fed-batch process [7]. CAFNN is similar to a multi-layer feed-forward neural network but
123
426
has its own special structure. The network can be trained by a modified Levenberg–Marquardt (LM) algorithm, and it offers an effective and simple optimal control strategy, in which the gradient information can be computed analytically in a recursive fashion. However, the neural network model developed is only an approximation of the fed-batch process and its prediction offsets may occur due to modelplant mismatches and unknown disturbances [7]. Then optimal control policy calculated from the CAFNN model may not be optimal when applied to the actual process. Several methods have been proposed to address this issue. Xiong and Zhang [6] propose a re-optimization scheme for batch processes to overcome the detrimental effects of model-plant mismatches and unknown disturbances. Zhang [8] presents a reliable optimization method for batch process based on bootstrap aggregated neural network models by incorporating confidence bounds of those model prediction into the optimization objective function. The repetitive nature of fed-batch processes allows the information of previous batch runs to be used to improve the performance of next batch runs in the fashion of iterative learning control (ILC) [9–11]. The basic idea of ILC is to update the control trajectory for a new batch run using the information from previous batch runs so that the output trajectory converges asymptotically to the desired reference trajectory. Refinement of control signals based on ILC can significantly enhance the performance of tracking control systems. Optimal ILC is one of important methods for designing an iterative learning law, in which a quadratic objective function is to be optimized [12]. Lee et al. in several related articles [10, 13, 14] proposed the quadratic criterionbased ILC (Q-ILC) approach for tracking control for temperature of batch processes based on a linear time varying (LTV) tracking error transition model. In our previous work [11], an ILC strategy for the tracking control of product quality in batch processes based on an LTV perturbation model was proposed. The convergence of ILC law based on the batch-wise LTV perturbation model was also demonstrated. To address the problem of model-plant mismatches, the model prediction errors in the previous batch run are added directly to the model predictions for the current batch run. Since ILC is well developed for linear models, most of the ILC-based batch-to-batch control schemes are based on some kinds of linear models (e.g., [9, 12, 13]). Nonlinear model based ILC schemes for batch and fed-batch processes have also recently been proposed [6, 15]. Xiong and Zhang [6] present a recurrent neural network based ILC scheme for batch processes where the prediction errors of recurrent neural network are filtered from previous batches and added to the model predictions for the current batch, and then optimization is performed based on the updated predictions. Zhang [15] presents an ILC scheme based on a feed-forward neural network model where the network is linearized
123
Neural Comput & Applic (2008) 17:425–432
around the current batch. Based on the linearized model, the control policy for next batch is updated using ILC to minimize the tracking errors at the end of the next batch. In this study, CAFNN-based ILC strategy is proposed to improve the tracking control performance for fed-batch processes. To improve the accuracy of model predictions, the model errors of the immediately previous batch are also used to modify the CAFNN model prediction. Because the gradient information can be calculated directly from its special structure, the linearized model of CAFNN is easy to be obtained analytically and then applied to the batch-to-batch ILC. Therefore, based on the batch-to-batch ILC law, the control policy for a next batch is modified from the previous control profile. This procedure is repeated from batch to batch, and then the performance of the tracking control of the endpoint product qualities can be improved gradually. The rest of this paper is structured as follows: the structure of CAFNN is presented in Sect. 2. In Sect. 3, the batch-to-batch ILC strategy for fed-batch processes based on a CAFNN model is described in detail, where the algorithm for the recursive computation of the gradient information is also presented. The proposed method is demonstrated on a simulated fed-batch ethanol fermentation process in Sect. 4. Finally Sect. 5 draws some concluding remarks.
2 Control-affine feedforward neural network The control affine feed-forward neural network (CAFNN) model [7] is reviewed in this section. In many biochemical processes, the reactors are operated in fed-batch mode, where the feed rate is used for control, as shown in Fig. 1. In such a fed-batch process, the general interests are concerned with determining the
Fig. 1 A diagram of fed-batch process
Neural Comput & Applic (2008) 17:425–432
427
feeding policy to the reactor that will give the maximum amount of the desired product. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control problem, considerable difficulties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because fed-batch processes are frequently nonlinear and state constrained [16, 17]. In this study, we consider that fed-batch processes usually have a fixed batch length (tf) and all batches run from the same initial condition. Fed-batch process usually can be considered as a class of control affine nonlinear systems, mathematically formulated as [7]: x_ k ¼ f ðxk Þ þ gðxk Þuk ;
xk ð0Þ ¼ xk0
ð1Þ
yk ¼ xk
ð2Þ
where the subscript k denotes the batch index, x [ Rp a vector of process states, u [ Rq a vector of process inputs, y [ Rp a vector of process output, respectively. It is assumed here that the duration of batch run consists of N sampling intervals (i.e., N = tf/h). From above control affine system (1) and (2) and considering its special characteristics, a discrete time model for the system can be described as yk ðt þ 1Þ ¼ f ðyk ðtÞ; . . .; yk ðt n þ 1ÞÞ þ gðyk ðtÞ; . . .; yk ðt n þ 1ÞÞuk ðtÞ
ð3Þ
where n (n p) is the order of the nonlinear system. To model this kind of nonlinear system, a special neural network, CAFNN, is proposed [7] and its topology is shown in Fig. 2. The dashed frame in Fig. 2 is consisted of an inner feed-forward neural network with a hidden layer, thus the CAFNN model can be considered similarly as a two-hidden-layer network. Model output of CAFNN is formulated as ^ k ðtÞÞ y^k ðt þ 1Þ ¼ W 0 ðtÞT Fðv ¼ f^ðvk ðtÞÞ þ g^ðvk ðtÞÞuk ðtÞ
ð4Þ
where vk ðtÞ ¼ ½yk ðtÞ; yk ðt þ 1Þ; . . .; yk ðt n þ 1ÞT ; W0(t) = ^ k ðtÞÞ ¼ ½f^ðvk ðtÞÞ; g^ðvk ðtÞÞT ; and f^ and g^ are [1, uk(t)]T, Fðv predictions of the inner feed-forward network, respectively. ^ k ðtÞÞ can be calculated further according to the Fðv topology of feed-forward network ^ k ðtÞÞ ¼ Fðv
nh X m¼1
w2jm a
nI X
!! w1ml vk ðlÞ
þ
l1m
þ l2j
Fig. 2 The structure of control affine feed-forward neural network
model, and a(z) = 1/(1 + ez), respectively. For a CAFNN with nI inputs and nh hidden neurons, its structure can be symbolized as CAFNN: nI-nh-21. There are some differences between a CAFNN and a multi-layer feed-forward neural network (MFNN) with two-hidden-layers and two hidden neurons on the second hidden layer. To model a fed-batch process, the input vector of such an MFNN is v(t) = [y(t),…,y(tn + 1), u(t)]T, which is symbolized as MFNN: (nI + 1)-nh-21, but the input vector of CAFNN only includes values of output, i.e., v(t) = [y(t),…,y(tn + 1)]T. A CAFNN can be trained using a number of training methods, such as the back-propagation method, the conjugate gradient method, Levenberg–Marquardt (LM) optimization, or methods based on genetic algorithms. In this paper, CAFNN is trained by using a modified LM algorithm with regularization presented in our previous work [7]. After the CAFNN is trained, the network can predict recursively the endpoint output y^k ðNÞ for the kth batch given the initial conditions (u0, y0) and the whole control profile. The model prediction error ek(N) for the kth batch is then calculated by ek ðNÞ ¼ yk ðNÞ y^k ðNÞ
ð6Þ
where yk(N) is the actual product quality values at the end of a batch. To improve the accuracy of model predictions, the model errors of the immediately previous batch are used to modify the CAFNN model prediction: y~k ðNÞ ¼ y^k ðNÞ þ ek1 ðNÞ
ð7Þ
where y~k ðNÞ is the modified prediction of endpoint product quality values.
ð5Þ
l¼1
where j = 1, 2, w1ml and w2jm are connection weights, l1m and l2j are the thresholds, nI and nh are the numbers of nodes in the input layer and hidden layer of the inner NN
3 Batch-to-batch control of fed-batch processes In fed-batch processes, the maximum amount of the desired product is usually considered by determining the optimal
123
428
Neural Comput & Applic (2008) 17:425–432
feed rate to the reactor. Based on the CAFNN model developed above, the optimal control policy U can be obtained by solving the following optimization problem [7] min /ð yðNÞÞ U
ð8Þ
where / is a scalar objective function in terms of the endpoint product qualities. Due to the CAFNN model, above optimization problem is usually solved using a nonlinear programming method such as the sequential quadratic programming (SQP) method [7], iterative dynamic programming (IDP) method [16, 17], and so on. As the CAFNN only approximates the fed-batch processes, prediction offsets are unavoidable due to model-plant mismatches. Then optimal control policy calculated from the CAFNN model may not be optimal when applied to the actual process [6, 7]. However, the repetitive nature of fed-batch processes allows the information of previous batch runs to be used in order to improve the performance of next batch runs [10, 15]. In this study, batch-to-batch ILC strategy is proposed to improve the control performance and limit the deterioration. Let us define the control and product quality sequences through a whole batch duration as Uk ¼ ½uTk ð0Þ; uTk ð1Þ; . . .; uTk ðN 1ÞT Yk ¼ ½yTk ð1Þ; yTk ð2Þ; . . .; yTk ðNÞT :
ð9Þ ð10Þ
The aim of batch-to-batch control is to use model-based ILC to find an updating mechanism for the control sequence of the feed rate Uk+1 of a new batch so that the measured endpoint product qualities yk+1(N) converges asymptotically towards the desired values yd(N). Using the modified prediction errors upon completion of the kth batch to update the control policy for the (k + 1)th batch, the following quadratic objective function is considered [9, 15] Jkþ1 ¼
i 1h ekþ1 ðNÞ þ DUTkþ1 RDUkþ1 e~kþ1 ðNÞT Q~ 2
ð11Þ
where e~kþ1 ðNÞ is the tracking error of the endpoint value e~kþ1 ðNÞ ¼ yd ðNÞ y~kþ1 ðNÞ
ð13Þ
and Q and R are weighting matrices with appropriate dimensions, respectively.
123
N 1 X oyðNÞ y^kþ1 ðNÞ ¼ y^k ðNÞ þ ouðiÞ i¼0
ðukþ1 ðiÞ uk ðiÞÞ þ dkþ1
uk ðiÞ
y^k ðNÞ þ GTk DUkþ1
ð14Þ
where Gk ¼
oyk ðNÞ oyk ðNÞ oyk ðNÞ ; ; . . .; ouk ð0Þ ouk ð1Þ ouk ðN 1Þ
T ð15Þ
and dk+1 is the modeling error due to the linearization. It is reasonable to assume here that dk+1 can be omitted by neglecting the higher-order terms. According to the special structure of CAFNN in (5), the gradient information rJ(u(k)) can be computed directly. Let us define oyk ðNÞ of^ðyk ðNÞÞ o^ gðyk ðNÞÞ ¼ þ uk ðN 1Þ: oyk ðN lÞ oyk ðN lÞ oyk ðN lÞ
ð16Þ
Then the gradient information qyk/quk can be computed by 8 i¼N1 g^ðyk ðNÞÞ; oyk ðNÞ < P n oyk ðNÞ oyk ðNlÞ ¼ ð17Þ i\N 1 : : oyk ðNlÞ ouk ðiÞ ; ouk ðiÞ l¼0
It should be noted that in above (17) the following partial derivatives are zeros
ð12Þ
DUk+1 is the difference of control sequences between two adjacent batches DUkþ1 ¼ Ukþ1 Uk
In order to limit the deterioration of control performance due to model-plant mismatches and unknown disturbances, a linearized model around the neural network model is obtained. In many batch and fed-batch processes, subtracting the time-varying nominal trajectories from the process operation trajectories removes the majority of the process nonlinearity and allows linear modeling methods to perform well on the resulting perturbation variables [18]. The linearization of nonlinear fed-batch processes around the nominal trajectory is a good approximation to the real processes as the input change is typically small. According to the CAFNN model, the prediction for the (k + 1)th batch can be approximated using the first order Taylor series expansion based on the kth batch:
oyk ðN lÞ ¼ 0; ouk ðiÞ
i N l; ðl ¼ 1; . . .; nÞ:
ð18Þ
By computing qyk(j)/quk(i) recursively from j = i + 1 to j = N, the gradients qyk(N)/quk(i), (i = 0, 1,…, N1) can be obtained. The recursive algorithm for computing the gradient qyk(j)/ quk(i) is outlined as follows [7]
Neural Comput & Applic (2008) 17:425–432
8 g^ðyk ðjÞÞ; oyk ðjÞ < P n oyk ðjÞ oyk ðjlÞ ¼ ouk ðiÞ : oyk ðjlÞ ouk ðiÞ ;
429
i¼j1 i\j 1
ð19Þ
l¼0
ekþ1 ðNÞ ¼ yd ðNÞ y^kþ1 ðNÞ þ ekþ1 ðNÞ ¼ yd ðNÞ y~kþ1 ðNÞ ek ðNÞ ekþ1 ðNÞ ¼ e~kþ1 ðNÞ Dekþ1 ðNÞ:
ð29Þ
oyk ðjÞ of^ðyk ðjÞÞ o^ gðyk ðjÞÞ ¼ þ uk ðj 1Þ oyk ðj lÞ oyk ðj lÞ oyk ðj lÞ
ð20Þ
Then above batch-to-batch control law (26) can be calculated further by
nh of^ðyk ðjÞÞ X ¼ w2 dðv ðjÞÞw1ml oyk ðj lÞ m¼1 1m k
ð21Þ
Ukþ1 ¼ Uk þ Kk ek ðNÞ:
ð30Þ
Substituting (29) to (28), we have
nh X
o^ gðyk ðjÞÞ ¼ w2 dðv ðjÞÞw1ml oyk ðj lÞ m¼1 2m k
ð22Þ
ekþ1 ðNÞ ¼ I GTk Kk ek ðNÞ Dekþ1 ðNÞ:
dðvk ðjÞÞ ¼ Oðvk ðjÞÞð1 Oðvk ðjÞÞÞ
ð23Þ
Theorem 1 Let {ek+1(N)} be the tracking error sequence under the above optimal ILC law (30). If the following conditions are satisfied, V k = 1, 2,...,?
ð24Þ
I GT Kk k\1; k
Oðvk ðjÞÞ ¼ a
nI X
! w1ml yk ðj lÞ þ l1m
l¼1
where l = 1,..., n, i = 0, 1,..., N1, j = 1,..., N. In above equations, yk are predictions of CAFNN. Actually the measured product qualities can also be used to correct the gradient information as a feedback during batch-to-batch control. Substituting (7) and (14) to (12), an iterative relationship for e~kþ1 ðNÞ along the batch index k can be obtained e~kþ1 ðNÞ ¼ yd ðNÞ y^k ðNÞ þ GTk DUkþ1 ek ðNÞ
GTk DUkþ1 ek ðNÞ
ð26Þ
where Kk is the learning rate in the ILC 1 Kk ¼ Gk QGTk þ R Gk Q:
ð27Þ
Substituting (26) to (25), we have e~kþ1 ðNÞ ¼ I GTk Kk ðe~k ðNÞ Dek ðNÞÞ:
then kekþ1 ðNÞk will converge to a small value. Proof Using (31) recursively, the following relationships for ek+1(N) can be obtained ekþ1 ðNÞ ¼ I GTk Kk ek ðNÞ Dekþ1 ðNÞ ¼ I GTk Kk I GTk1 Kk1 ek1 ðNÞ I GTk Kk Dek ðNÞ Dekþ1 ðNÞ
i¼0
ð25Þ
where Dek ðNÞ ¼ ek ðNÞ ek1 ðNÞ is the prediction difference between two adjacent batches. Setting qJ/ qUk+1 = 0 in the objective function (11) and through straightforward computation, the batch-to-batch ILC law can be calculated as DUkþ1 ¼ Kk ðe~k ðNÞ Dek ðNÞÞ
ð32Þ
¼ " # k Y T ¼ I G i K i e0
¼ yd ðNÞ ðy~k ðNÞ ek1 ðNÞÞ
¼ e~k ðNÞ GTk DUkþ1 Dek ðNÞ
kDek ðNÞk E\1
ð31Þ
ð28Þ
The tracking error of the endpoint value is defined as ek+1(N) = yd(N) yk+1(N). From above definitions of ek(N) and Dek(N), it holds
" # ! k k
Y X T I Gj Kj Dei ðNÞ Dekþ1 ðNÞ i¼1
j¼i
ð33Þ where e0 is the initial tracking error and is assumed to be obtained, G0 is the initial model obtained from the CAFNN, and K0 is set to 0, which represents that the initial learning rate in ILC law is not calculated, respectively. It is derived from (33) that " # Y k I GTi Ki e0 kekþ1 ðNÞk i¼0 " # ! X k k
Y T þ I Gj Kj Dei ðNÞ i¼1 j¼i þ kDekþ1 ðNÞk: ð34Þ Considering I GTi Ki k\1; it holds as follows
123
430
Neural Comput & Applic (2008) 17:425–432
" # Y k T I Gi Ki e0 i¼0
k Y I GT Ki ke0 k ke0 kkk : i
ð35Þ
i¼0
Furthermore, considering kDei ðNÞk E; it also holds " # ! X k k
Y T I Gj Kj Dei ðNÞ þ kDekþ1 ðNÞk i¼1 j¼i " # k k Y X I GTj Kj kDei ðNÞk þ kDekþ1 ðNÞk i¼1
j¼i
k k Y X E I GTj Kj i¼1
" E
j¼i
k k X Y i¼1
! k
!
#
V low ½1; 1; . . .; 1Ukþ1 V hi :
ð40Þ
4 Application to a simulated ethanol fermentation process
þ1
j¼i
ð36Þ
Substituting (35) and (36) to (34), we have kekþ1 ðNÞk ke0 kkk þ
Eð1 kkþ1 Þ : 1k
ð37Þ
Then the following convergence of ek+1(N) can be obtained lim kekþ1 ðNÞk ¼
E ¼ g: 1k
ð38Þ
It means that the tracking error of endpoint value ek+1(N) converges to a small value with respect to the batch index. h Remark According to (27), Kk is related to parameters Q and R. These two weighting matrices can be set specifically so that the condition I GTk Kk k\1 will be satisfied for all batch runs, although the converge rate probably degrades. When the CAFNN model is properly trained and validated, the modeling error can be bounded by a small value and then D ek(N) is also bounded. Therefore, the conditions in (32) can be always satisfied. Based on the above ILC law (30), the control profile Uk+1 for the next batch is modified from the previous control profile Uk. This procedure is repeated from batch to batch, and then the performance of the tracking control of the endpoint product qualities can be improved gradually. In practice, in order to ensure safe and smooth operations of industrial processes, input (e.g., feed rate) and output variables (e.g., product qualities) of fed-batch process are always constrained [1, 6, 10]. For example, if there is no outflow of batch reactor, then the feed rate must be chosen so that the reactor volume does not exceed the
123
ð39Þ
þE
Eð1 k Þ : 1k
k!1
Ulow Ukþ1 Uhi
Above constraints may become active, which usually prevents the convergence to zero tracking error during the batch-to-batch control.
kþ1
¼
physical volume of the reactor. The feed rate may also be constrained to ensure smooth operations. These input and output constraints should be satisfied if they exist. For simplicity of this study on the fed-batch processes, constraints on the magnitude values of control variable and their sum value are only considered as follows
Consider an ethanol fermentation process as shown in Fig. 1, whose mechanistic model in the form of differential algebraic equations (DAE) is described as follows [19]: dx1 x1 ¼ Cx1 u dt x4
ð41Þ
dx2 ð150 x2 Þ ¼ 10Cx1 þ u x4 dt
ð42Þ
dx3 x3 ¼ Dx1 u dt x4
ð43Þ
dx4 ¼u dt
ð44Þ
with C¼
0:408x2 ð1 þ x3 =16Þð0:22 þ x2 Þ
ð45Þ
D¼
x2 ð1 þ x3 =71:5Þð0:44 þ x2 Þ
ð46Þ
where x1 is the cell mass concentration, x2 is the substrate concentration, x3 is the product concentration, and x4 is the liquid volume of the reactor. x4 is limited by the 200 l vessel size. The initial condition is specified as x0 = [1, 150, 0, 10]T. The batch length tf is fixed to be 63.00 hours and divided into N = 10 equal stages (i.e., sampling time h = tf/N = 6.3). The feed rate into the reactor u is used for control and constrained by 0 u 12 (l/h). There is no outflow, so the feed rate must be chosen so that the batch volume does not exceed the physical volume of the reactor. Since x4 is obtained by integrating u, the constraint on u should also be satisfied
Neural Comput & Applic (2008) 17:425–432
431 12 1st batch 5th batch 15th batch
10 8 Feed rate ( L/h )
with [1,1,...,1] Uk 190. The performance index is to maximize the amount of the desired production at the end of batch. The fermentation process was formulated and used for optimal control studies by Hong [19]. This typical biochemical system exhibiting singular control was also studied for global optimization by Hartig [16] and by Bojkov [17]. In this study, the above mechanistic model (41)–(46) is assumed to be not available. Thus a CAFNN based model has to be utilized to model the nonlinear relationship between y = x3 and u, and the CAFNN model is described as
6 4 2 0
yk ðt þ 1Þ ¼ f^ðyk ðtÞ; yk ðt 1Þ; yk ðt 2ÞÞ þ g^ðyk ðtÞ; yk ðt 1Þ; yk ðt 2ÞÞuk ðtÞ:
0
10
20
ð47Þ
For CAFNN modeling, 52 batches of the process operation under different feeding policies were simulated from the mechanistic model to generate the data sets. Then the data sets are divided into training data of 30 batches, testing data of 20 batches, and unseen validation data of 2 batches. After cross-validation, the appropriate topology of CAFNN was selected with 3 input neurons and 10 hidden neurons, i.e., 3-10-2-1. After the CAFNM is trained, the long-range predictions from this CAFNN model on the unseen batch data are shown in Fig. 3. It is clear that the CAFNN model has captured the dynamic trends of the product quality and can be considered as being accurate enough to model the fedbatch fermentation process. For batch-to-batch control, the parameters of ILC were set as Q = I and R = 0.1 IN, and the desired output yd(N) is set to 103.53 [7]. To investigate the performance of the proposed control strategy, the optimal value of the final product concentration is first obtained by solving the
30 time ( h )
40
50
60
Fig. 4 Trajectories of feed rate Uk under the batch-to-batch control
optimization problem based on the CAFNN model, in which the sequential quadratic programming (SQP) method is used to solve the nonlinear optimization problems. Due to CAFNN model-plant mismatches, the value drops to 95.05, an 8.2% reduction. To carry out batch-to-batch control scheme, the calculated control profile only based on the CAFNN is used as the control policy of the first run of the ILC method, as shown in Fig. 4. After 15 batch runs, the actual product concentration is improved to 101.42, only 2.1% reduction, as shown in Fig. 5. Figure 5 shows the root-mean-square-error (RMSE) of tracking error of product quality ek(N) at the batch endpoint under the proposed batch-to-batch control strategy. It can be seen that ek(N) is improved gradually and converges after almost 6 batch runs, which means that the endpoint 9
ymdl(N) = 103.7319 ymCAFNN(N) = 98.7852
8
110
7
90
6
80
3
Product Concentration x ( g/L )
100
70
5
60
4
50 40
3
30
simulated process CAFNN prediction
20
2
10
1 1
0 0
10
20
30 time ( h )
40
50
60
Fig. 3 Predictions of the CAFNN model on the unseen batch data
3
5
7 9 batch number
11
13
15
Fig. 5 Tracking performance of RMSE of ek(N) under the batch-tobatch control
123
432
Neural Comput & Applic (2008) 17:425–432 Acknowledgments The work is partially supported by National Natural Science Foundation of China under grant 60404012, Proj 863 (No. 2007AA04Z193), SRF for ROCS of SEM of China, New Star of Science and Technology of Beijing City, and IBM China Research Lab 2007 UR-Program.
200 180
1st batch 5th batch 15th batch
160 Volume ( L )
140 120
References
100 80 60 40 20 0
0
10
20
30 time ( h )
40
50
60
Fig. 6 Trajectories of reactor volume
value converges asymptotically to the desired one. It should be noted that there are small fluctuation of ek(N) due to the constraints on the feed rate. Figures 4 and 6 show, respectively, the feed rate (control variable) profiles uk and the volume of batch reactor of the 1st, 5th, and 15th batch runs. It can be seen that in Fig. 3 the constraints on the magnitude control values are not violated under the batch-to-batch control. But the constraints on their sum value, i.e., liquid volume of the reactor, are active as shown in Fig. 6. If the sum of control sequence at time i is satisfied with the limit, then the control values after time i must be set to zero, as shown in Fig. 4.
5 Conclusions A batch-to-batch iterative learning control strategy for fedbatch processes is proposed based on control affine feedforward neural network. The CAFNN has its own special structure and is very effective to model fed-batch process. The gradient information can be computed analytically from CAFNN and then applied to the ILC method. By using the information of previous batches, the endpoint product qualities of fed-batch processes are improved gradually from batch to batch. The proposed control scheme is illustrated on a simulated fed-batch ethanol fermentation process. The simulation results show that the proposed method takes effects of the tracking performance. The deterministic model without noises in the fed-batch processes is only studied here. In future works, the fedbatch processes in the presence of model uncertainties will be considered further.
123
1. Bonvin D (1998) Optimal operation of batch reactors—a personal view. J Process Contr 8:355–368 2. Park S, Ramirez WF (1988) Optimal production of secreted protein in fed-batch reactors. AIChE J 34:1550–1558 3. Luus R (1992) On the application of iterative dynamic programming to singular optimal control problems. IEEE Trans Autom Control 37:1802–1806 4. Sjoberg J, Zhang Q, Ljung L, Benveniste A et al (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31:1691–1724 5. Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop PJ (1992) Neural networks for control systems—a survey. Automatica 28:1083– 1112 6. Xiong ZH, Zhang J (2005) A batch-to-batch iterative optimal control strategy based on recurrent neural network models. J Process Contr 15:11–21 7. Xiong ZH, Zhang J (2004) Modelling and optimal control of fedbatch processes using a novel control affine feedforward neural network. Neurocomputing 61:317–337 8. Zhang J (2004) A reliable neural network model based optimal control strategy for a batch polymerisation reactor. Ind Eng Chem Res 43:1030–1038 9. Bristow DA, Tharayil M, Alleyne AG (2006) A survey of iterative learning control. IEEE Control Syst Mag 26:96–114 10. Lee JH, Lee KS, Kim WC (2000) Model-based iterative learning control with a quadratic criterion for time-varying linear systems. Automatica 36:641–657 11. Xiong ZH, Zhang J (2003) Product quality trajectory tracking of batch processes using iterative learning control based on time varying perturbation model. Ind Eng Chem Res 42:6802–6814 12. Amann N, Owens DH, Rogers E (1996) Iterative learning control for discrete-time system with exponential rate of convergence. IEE Proc Control Theory Appl 143:217–224 13. Lee KS, Lee JH (2000) Convergence of constrained model-based predictive control for batch processes. IEEE Trans Autom Control 45:1928–1932 14. Lee KS, Lee JH (2003) Iterative learning control-based batch process control technique for integrated control of end product properties and transient profiles of process variables. J Process Contr 13:607–621 15. Zhang J (2005) A neural network based strategy for the integrated batch-to-batch control and within batch control of batch processes. Trans Inst Meas Control 27:391–410 16. Hartig F, Keil FJ, Luus R (1995) Comparison of optimization methods for a fed-batch reactor. Hung J Ind Chem 23:141–148 17. Bojkov B, Luus R (1996) Optimal control of nonlinear systems with unspecified final times. Chem Eng Sci 51:905–919 18. Russell SA, Kesavan P, Lee JH (1998) Recursive data-based prediction and control of batch product quality. AIChE J 44:2442– 2458 19. Hong J (1986) Optimal substrate feeding policy for fed batch fermentation with substrate and product inhibition kinetics. Biotechnol Bioeng 27:1421–1431
