stochastic analysis and control of real-time ... - Semantic Scholar
STOCHASTIC ANALYSIS AND CONTROL OF REAL-TIME SYSTEMS WITH RANDOM TIME DELAYS Johan Nilsson, Bo Bernhardsson and Björn Wittenmark Department of Automatic Control Lund Institute of Technology Box 118, S-221 00 Lund, Sweden [email protected]
Abstract: The paper discusses modeling and analysis of real-time systems subject to
random time delays in the communication network. A new method for analysis of existing schemes is presented. The method is used to evaluate di erent suggested schemes from the literature. A new scheme for handling the random time delays is then developed and successfully compared with previous schemes. The new scheme is based on stochastic control theory and a separation property is shown to hold for the optimal controller.
Keywords: Delay compensation, distributed computer control systems, real-time systems, stochastic control, stochastic parameters. 1. INTRODUCTION Many real-time systems are implemented as distributed control systems, where the control loops are closed over a communication network. There will inevitably be time delays in the communication net. As long as the sampling periods are long compared with these delays there is no need to consider the in uence of the delays. As the demand on the control system increases it will be more and more important to take the delays into account in the analysis and the design of the control system. While inaccuracies, disturbances, etc, have been extensively studied in the control literature the timing problems in real-time systems have just recently attracted attention, and in the communication literature the feedback control aspect has not been treated extensively. This is thus an area where much can be gained by combining ideas from the two elds of control and real-time systems. Several problem formulations have been suggested by previous authors. A general setup would involve a dis-
This
work is supported by NUTEK, Swedish National Board for Industrial and Technical Development, Project Dicosmos, 933485.
tributed control structure, where communication between di erent control system nodes is achieved over a communication network. Often a centralized controller is used and the actuators and sensors are communicating with the controller(s) via a bus. The e ect of communication delays is discussed, for instance, in Ray (1989), Shin and Kim (1992), and Krtolica (1994). For an introduction to the area with remarks on open problems see Wittenmark (1995). et al.
et al.
We will analyze a simple structure with just one controller and one process connected as in Figure 1. A number of previous authors have suggested such a control scheme with slightly di erent setups. One can consider several cases, depending on how the actuator, sensor, and controller nodes are synchronized. The scheme we choose to analyze has a time driven sensor system sampled with a constant sampling rate and event-driven controller and actuator nodes. This means that a transmitted signal is used as soon as it arrives to the controller or actuator node. There are essentially three kinds of computer delays in the system:
Communication delay between the sensor and the controller sc Computational delay in the controller c Communication delay between the controller and the actuator ca The control delay for the control system, in principle, equals the sum of these delays. In this paper we will look at the in uence of the delays sc and ca . The e ect of c can be embedded in ca. In the article we will compare with a controller structure where bu ers are used at the controller node and at the actuator node. This is a scheme suggested by Luck and Ray (1990). For other approaches see Krtolica (1994), Ray (1994), and Voulgaris (1994). We also compare with the standard LQG-controller that neglects the e ects of the time delays. It is of interest to be able to analyze all di erent suggested control schemes and to make appropriate trade-o s between complexity and performance. The essential problem in the analysis is that the time delays are varying in a random fashion. In this paper we will formulate and solve a problem where the performance of such systems can be analyzed. The methodology is based on a stochastic description of the variations of the delays. The in uence of the stochastic variation of the time delays can be computed under di erent assumptions. Formulas suited for numerical calculations are derived in the paper. The main assumption in this paper is that the time delays are statistically mutually independent. This restriction is discussed in the paper and will be the topic for future research. A new improved scheme is also presented. It is based on optimal stochastic control. It is assumed that the probability distribution of the time delays are known and that the control and measurement signals are supplemented with so called \time stamps", the time when the signal was generated. The control law satis es a separation property when time delays are uncorrelated. A suboptimal scheme is also suggested. This scheme requires less computation but seems to give results close to optimal. The theoretical results are compared with Monte-Carlo simulations. The numerical results support the theoretical claims and show the possible advantage of being able to make an exact computation of the in uence of the random time delays.
h Actuator node
ca
Network
sc
Controller node Fig. 1.
Digital control system with induced delays.
Process output
et al.
2. PROBLEM FORMULATION We will make the following assumptions about the control system:
Sensor node
Process
yk ksc yk kca uk
Process input Fig.
uk
(k ? 1)h (k + 1)h kh Timing of signals in the control system. The rst diagram illustrates the process output and the sampling instants, the second diagram illustrates the signal into the controller node, the third diagram illustrates the signal into the actuator node, and the fourth diagram illustrates the process input, compare with Figure 1. 2.
The output of the process is sampled periodically
without any scheduling disturbances. The sampling period is h. The communication delays sc and ca are randomly varying. Consecutive time delays are independent and their probability distributions are known a priori. The control signal is applied to the process as soon as the data arrives at the actuator node. The total time delay sc + ca have arbitrary distribution, but is always less than one sampling period. The sizes of the past time delays are known to the controller. One way to achieve this is by marking every signal that is transferred in the system with a time stamp.
The timing of signals in the control system is illustrated in Figure 2. The process to be controlled is assumed to
be of the form
x_ = Ax + Bu + Gv where x is the process state, u the input and v is white noise with unit incremental variance. Discretizing the process in the sampling instants kh taking into account the e ect of the time delays ksc and kca, see Figure 2, gives the discrete time model xk+1 =xk + ?0 ( ksc ; kca)uk + ?1( ksc ; kca)uk?1 + vk (1) yk =Cxk + ek (2) where =eAh ?0( ksc ; kca) = ?1( ksc ; kca) =
Z h? ksc ? kca 0
Zh
h? ksc ? kca
eAs dsB
eAs dsB
and vk and ek are uncorrelated white noise with zero mean and covariance matrices R1 and R2 respectively. Denote the information available when control signal uk is calculated by Yk . This has the structure
Yk = yk ; yk?1; :::; ksc; ksc?1; ::::; kca?1; kca?2; :::; uk?1; uk?2; ::: : Notice that the sensor to controller delay sc at time k and older are available, and that the controller to actuator delays ca are assumed known up to time k ? 1. We assume that the control signal is a function of all information available when it is calculated, i.e. uk = f (Yk ).
3. EVALUATION OF SCHEMES It turns out that with the proposed control schemes the closed loop system can be written as zk+1 = A( k )zk + B ( k )wk where k is a random process uncorrelated with wk , zk is a state vector for the closed loop system and wk is a vector with independent white noise components with unit variance. For example zk can be a vector with xk , uk?1, and the controller state. Similarly k can be a vector consisting of ksc and kca . Often zk has the property that xk and uk can be obtained by a linear transformation of zk , such that xk uk = Q s zk :
One way to compare the performance of di erent control schemes when subjected to random communication delays is to set up a cost function to be minimized by the controller. For the LQG-case it is of interest to evaluate the cost function
xk T xk Q u E ;w uk k = ;w E zkT QTs QQszk = tr QTs QQsTk (3)
where Tk = ;w E zk zkT . To evaluate the quantity Tk , which is independent of Q and Qs , we use that Tk+1 = ;w E [Ak zk + Bk wk ][Akzk + Bk wk ]T = E [Bk BkT + Ew Ak zk zkT ATk ]: Using Kronecker products this can be written as B B T ) + Ek (Ak Ak ) vec(Tk ): (4) vec(Tk+1 ) = vec(E k k k
The short forms Ak = A( k ) and Bk = B ( k ) are used to simplify reading. We have here used that zk and wk are independent and that zk is independent of k . This is crucial for the applied technique to work and indirectly requires that k and k?1 are independent. From the above calculations it follows a direct algorithm to calculate the stationary cost function in the LQGcase. Algorithm 1|Stationary cost function
1. Iterate (4) forward in time to get the stationary value T1 of Tk . 2. Calculate the stationary cost function from (3). Stochastic Stability From (4) it is seen that stochastic stability of the closed loop system is determined by the stability of the matrix (5) E (A( k ) A( k )): k We have now derived formulas for evaluation of cost functions from equations for the corresponding closed loop system. In the case of quadratic cost function an algorithm for evaluation of the mean cost has been found. A condition for test of stochastic stability followed from the calculations. 4. OPTIMAL STOCHASTIC CONTROL In this section we solve the control problem set up by the cost function N x T x X k k (6) JN = E Q uk k=0 uk
The solution of this problem follows by the same technique as for the standard LQG problem. We have the following result: Theorem 1|Optimal state feedback
Given the plant (1), with noise free measurement of the state vector xk , i.e. yk = xk . The control law that minimizes the cost function (6) is given by x k 22 ? 1 T 21 23 ~ ~ ~ uk = ?(Q22 + Sk+1 ) [ Q12 + Sk+1 Sk+1 ] uk?1 (7) where S~k+1 ( sc ) = Eca GT ( sc; ca )Sk+1 G( sc; ca) ?0( sc ; ca) ?1( sc ; ca) G( sc; ca ) = 0 I 0 T sc sc Sk = Esc fF1 ( )QF1( ) + F2T ( sc )S~k+1 ( sc )F2( sc )g # " ~22 ( Q 0 22 + Sk +1 ) sc 22 ? 1 F1( ) =(Q22 + S~k+1) ?(QT12 + S~k21+1) ?S~k23+1 sc F2( sc ) = F1 ( ) [0 I ] SN =0: S~kij is block (i; j ) of S~k ( sc ), and Qij is block (i; j ) of Q. Proof sketch Introduce a new state variable zk = xk uk?1 . Using dynamic programming with Sk the cost to go at time k, and with k the part of the cost function
that cannot be a ected by control, gives
zkT Sk zk + k
(
xk uk
T
)
x Q uk + zkT+1 Sk+1 zk+1 k k k + k+1 ( T xk Q xk E = Esc min ca uk uk k uk k ;vk o T Sk+1 zk+1 sc + k+1 + z k ( T k+1 x x k = Esc min Q k uk uk k uk 39 2 3T 2 xk > xk > = 7 6 7 6 ~ + 4 uk 5 Sk+1 4 uk 5> + tr Sk11+1 R1 + k+1: ; uk?1 > uk?1
E = min uk sc ; ca ;vk
The second equality follows from the fact that ksc is known when uk is determined. The third equality x follows from independence of k and kca , and from uk the de nition of S~k+1 . The resulting expression is a quadratic form in uk . Minimizing this with respect to uk gives the optimal control law (7). Theorem 1 states that the optimal controller with full state information is a linear ksc-depending feedback from the state and the previous control signal, i.e. x k sc uk = ?L( k ; Sk+1) : uk?1 The equation involved in going from Sk+1 to Sk is a stochastic Riccati equation evolving backwards in time. Each step in this iteration will contain expectation calculations with respect to the unknown ksc and kca . Under reasonable assumptions a stationary value S1 of Sk can be found by iterating the stochastic Riccati equation. In practice a tabular for L( ksc ; S1 ) can then be calculated to get a control law on the form xk sc uk = ?L( k ) u k?1 where L( ksc ) is interpolated from the tabular values of L( ksc ; S1 ) in real-time. In many cases the assumption of full state information does not hold. This can be solved by constructing a state estimate from the available data. In our setup there is the problem of the random time delays which enter in a nonlinear fashion. The fact that the old time delays up to time k ? 1 are known at time k, however, allows the standard Kalman lter of the process state to be optimal: Theorem 2|Optimal state estimate
Given the plant (1)-(2). The estimator
x^kjk = x^kjk?1 + K k (yk ? C x^kjk?1) with
x^k+1jk = ^xkjk?1 + ?0( ksc ; kca)uk + ?1 ( ksc ; kca)uk?1 + Kk (yk ? C x^kjk?1) x^0j?1 = E(x0 ) Pk+1 = Pk T + R1 ? Pk C T [CPk C T + R2]?1CPk P0 = R0 = E(x0xT0 ) Kk = Pk C T [CPkC T + R2]?1 K k = Pk C T [CPkC T + R2]?1
(8)
minimizes the error variance Ef[xk ? x^k ]T [xk ? x^k ] j Yk g. The lter gain Kk is independent of sc and ca. Moreover the estimation error is Gaussian with zero mean and covariance Pk ? Pk C T [CPkC T + R2]?1CPk .
Proof This can be proved as in Anderson and Moore (1979).
The following theorem justi es use of the estimated state in the optimal controller. Theorem 3|Separation property
Given the plant (1){(2), with Yk known when the control signal is calculated. The controller that minimizes the cost function (6) is given by x^ uk = ?(Q22 + S~k22+1 )?1 [ QT12 + S~k21+1 S~k23+1 ] kjk uk?1 where S~k is calculated as in Theorem 1, and x^kjk is the minimum variance estimate from Theorem 2. Full detailed proofs of Theorems 1{3 are available in Nilsson (1996). 5. A SUBOPTIMAL SCHEME A drawback with the optimal scheme is the complicated state feedback matrix L( ksc ). An alternative to the optimal controller is the suboptimal controller x ^kjk p p (9) uk = ?L [ k ?k ] uk?1 where pk = e
A( ksc +E kca )
?pk =
Z ksc +E kca 0
eAs dsB
and L is the optimal state feedback vector in the delayfree setup. Ep kca is the mean value of kca . The operation p k x^kjk + ?k uk?1 can be seen as a prediction from the state estimate at time kh to a state estimate when the control signal is applied at the actuator. This controller requires less computations than the optimal controller in Section 4. In Section 6 this controller is compared with the optimal controller. 6. EXAMPLE Consider the following plant, both plant and design speci cations are taken from Doyle and Stein (1979), dx = 0 1 x + 0 u + 35 (10) dt ?61 1 ?3 ?4 y = [2 1] x +
where E[ (t)] = E[(t)] = 0 and E[ (t1 ) (t2 )] = E[(t1)(t2 )] = (t1 ? t2). The control objective is to minimize the cost function 1 Z T (xT H T Hx + u2 )dt J = E Tlim !1 T 0
p p
where H = 4 5 [ 35 1 ]. The sampling period for the controller is chosen as h = 0:05. This is in accordance with the rule of thumb that is given in Åström and Wittenmark (1990). The time delays, ksc and kca , are assumed to be uniformly distributed on the interval [0; h=2] where 0 1. The stationary cost function will be evaluated and compared for four di erent schemes, an LQG-controller neglecting the time delays, the scheme with bu ers proposed in Luck and Ray (1990), the optimal controller derived in Section 4, and the suboptimal controller in Section 5. The rst design is done without taking any time delays into account. The process and the cost function are sampled to get discrete time equivalents, and the standard LQG-controller is calculated. This gives the design :911 T ; K = 2:690 ; K = 2:927 L = 38 ?4:484 8:094 ?5:012
This choice of L, K and K gives the following closed loop poles
sp( ? ?L) = f0:700 0:0702ig sp( ? KC ) = f0:743; 0:173g Even if these looks reasonable a Nyquist plot of the loop transfer function reveals a small phase margin, m = 10:9. The small phase margin indicates that there could be problems to handle unmodeled time delays. Numerical evaluation of (5) gives the stability limit crit = 0:249 for the controller neglecting the time delays. The scheme in Luck and Ray (1990) eliminates the randomness of the time delays by introduction of timed bu ers. This will, however, introduce extra time delay in the loop. The design for this scheme is done in the same way as in the standard LQG-problem. The third scheme we will compare is the optimal controller described in Section 4. Notice that the optimal state estimator gains K and K will be the same for the optimal controller as if the time delays were neglected. The feedback from the estimated state will have the form x^k sc uk = ?L( k ) u k?1
Control strategies when the control delay can be
4
5
x 10
Design neglecting time delays
4.5
larger than the sampling interval. A problem that occurs in this case is that there are then no guarantee that the samples arrive at the controller and the actuator in the order they were sent. A related problem is when samples may be lost in the communication, so called vacant sampling.
4
Scheme by Luck-Ray
3.5 3
Loss
2.5 2
8. REFERENCES
Suboptimal controller Optimal controller
1.5 1 0.5 0 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig. 3. Exact calculated performance (solid lines) of the four schemes, and simulated performance (dashed lines) of system (10). Notice the small di erence between the suboptimal controller and the optimal controller. For > 0:249 the controller neglecting the time delays fails to stabilize the process.
The suboptimal controller (9) uses L, K , and K from the standard LQG-controller. The stationary cost function has been evaluated for the four schemes with Algorithm 1. For comparison the stationary cost has also been evaluated by Monte Carlo simulation, which is made by calculating the mean cost during 4 104 simulated samples. The results agree very well, see Figure 3. From Figure 3 it is seen that the controller neglecting the time delays fails to stabilize the process for > crit. The optimal controller and the proposed suboptimal scheme outperforms the scheme proposed in Luck and Ray (1990). Note that for this example the cost is just slightly higher with the suboptimal controller than with the optimal controller. 7. CONCLUSIONS In this paper we have described a method to analyze performance of di erent schemes to compensate for randomly varying time delays. A condition for test of stochastic stability of the closed loop system has been derived. For the setup with time-driven sampling, event-driven controller, and event-driven actuator, an LQG-optimal controller has been found. The optimal controller has successfully been compared to a proposed suboptimal controller and some controllers proposed in the literature. Future work will include studies of
Optimal schemes when the time delays are correlated from sample to sample.
Anderson, B. and J. Moore (1979): Optimal ltering. Prentice-Hall, Englewood Cli s, N.J. Åström, K. J. and B. Wittenmark (1990): Computer Controlled Systems|Theory and Design. Prentice-Hall, Englewood Cli s, New Jersey, second edition. Doyle, J. C. and G. Stein (1979): \Robustness with observers." IEEE Trans. Automat. Contr., AC-24:4, pp. 607{611. Krtolica, R., U . Özguner, H. Chan, H. Göktas, J. Winkelman, and M. Liubakka (1994): \Stability of linear feedback systems with random communication delays." Int. J. Control, 59:4, pp. 925{953. Luck, R. and A. Ray (1990): \An observer-based compensator for distributed delays." Automatica, 26:5, pp. 903{ 908. Nilsson, J. (1996): Lic Tech thesis, Department of Automatic Control, Lund Institute of Technology, Lund, Sweden. Ray, A. (1989): \Introduction to networking for integrated control systems." IEEE Control Systems Magazine, January, pp. 76{79. Ray, A. (1994): \Output feedback control under randomly varying distributed delays." Journal of Guidance, Control, and Dynamics, 17:4, pp. 701{711. Shin, K. G. and H. Kim (1992): \Hard deadlines in realtime systems." In IFAC Symposium on Algorithms and Architectures for Real-Time Control, pp. 9{14, Seoul, Korea. Voulgaris, P. (1994): \Control of asynchronous sampled data systems." IEEE Trans. Automat. Contr., 39:7, pp. 1451{ 1455. Wittenmark, B., J. Nilsson, and M. Törngren (1995): \Timing problems in real-time control systems." In Preprints American Control Conference, pp. 2000{2004, Seattle, WA.
Abstract: The paper discusses modeling and analysis of real-time systems subject to
random time delays in the communication network. A new method for analysis of existing schemes is presented. The method is used to evaluate di erent suggested schemes from the literature. A new scheme for handling the random time delays is then developed and successfully compared with previous schemes. The new scheme is based on stochastic control theory and a separation property is shown to hold for the optimal controller.
Keywords: Delay compensation, distributed computer control systems, real-time systems, stochastic control, stochastic parameters. 1. INTRODUCTION Many real-time systems are implemented as distributed control systems, where the control loops are closed over a communication network. There will inevitably be time delays in the communication net. As long as the sampling periods are long compared with these delays there is no need to consider the in uence of the delays. As the demand on the control system increases it will be more and more important to take the delays into account in the analysis and the design of the control system. While inaccuracies, disturbances, etc, have been extensively studied in the control literature the timing problems in real-time systems have just recently attracted attention, and in the communication literature the feedback control aspect has not been treated extensively. This is thus an area where much can be gained by combining ideas from the two elds of control and real-time systems. Several problem formulations have been suggested by previous authors. A general setup would involve a dis-
This
work is supported by NUTEK, Swedish National Board for Industrial and Technical Development, Project Dicosmos, 933485.
tributed control structure, where communication between di erent control system nodes is achieved over a communication network. Often a centralized controller is used and the actuators and sensors are communicating with the controller(s) via a bus. The e ect of communication delays is discussed, for instance, in Ray (1989), Shin and Kim (1992), and Krtolica (1994). For an introduction to the area with remarks on open problems see Wittenmark (1995). et al.
et al.
We will analyze a simple structure with just one controller and one process connected as in Figure 1. A number of previous authors have suggested such a control scheme with slightly di erent setups. One can consider several cases, depending on how the actuator, sensor, and controller nodes are synchronized. The scheme we choose to analyze has a time driven sensor system sampled with a constant sampling rate and event-driven controller and actuator nodes. This means that a transmitted signal is used as soon as it arrives to the controller or actuator node. There are essentially three kinds of computer delays in the system:
Communication delay between the sensor and the controller sc Computational delay in the controller c Communication delay between the controller and the actuator ca The control delay for the control system, in principle, equals the sum of these delays. In this paper we will look at the in uence of the delays sc and ca . The e ect of c can be embedded in ca. In the article we will compare with a controller structure where bu ers are used at the controller node and at the actuator node. This is a scheme suggested by Luck and Ray (1990). For other approaches see Krtolica (1994), Ray (1994), and Voulgaris (1994). We also compare with the standard LQG-controller that neglects the e ects of the time delays. It is of interest to be able to analyze all di erent suggested control schemes and to make appropriate trade-o s between complexity and performance. The essential problem in the analysis is that the time delays are varying in a random fashion. In this paper we will formulate and solve a problem where the performance of such systems can be analyzed. The methodology is based on a stochastic description of the variations of the delays. The in uence of the stochastic variation of the time delays can be computed under di erent assumptions. Formulas suited for numerical calculations are derived in the paper. The main assumption in this paper is that the time delays are statistically mutually independent. This restriction is discussed in the paper and will be the topic for future research. A new improved scheme is also presented. It is based on optimal stochastic control. It is assumed that the probability distribution of the time delays are known and that the control and measurement signals are supplemented with so called \time stamps", the time when the signal was generated. The control law satis es a separation property when time delays are uncorrelated. A suboptimal scheme is also suggested. This scheme requires less computation but seems to give results close to optimal. The theoretical results are compared with Monte-Carlo simulations. The numerical results support the theoretical claims and show the possible advantage of being able to make an exact computation of the in uence of the random time delays.
h Actuator node
ca
Network
sc
Controller node Fig. 1.
Digital control system with induced delays.
Process output
et al.
2. PROBLEM FORMULATION We will make the following assumptions about the control system:
Sensor node
Process
yk ksc yk kca uk
Process input Fig.
uk
(k ? 1)h (k + 1)h kh Timing of signals in the control system. The rst diagram illustrates the process output and the sampling instants, the second diagram illustrates the signal into the controller node, the third diagram illustrates the signal into the actuator node, and the fourth diagram illustrates the process input, compare with Figure 1. 2.
The output of the process is sampled periodically
without any scheduling disturbances. The sampling period is h. The communication delays sc and ca are randomly varying. Consecutive time delays are independent and their probability distributions are known a priori. The control signal is applied to the process as soon as the data arrives at the actuator node. The total time delay sc + ca have arbitrary distribution, but is always less than one sampling period. The sizes of the past time delays are known to the controller. One way to achieve this is by marking every signal that is transferred in the system with a time stamp.
The timing of signals in the control system is illustrated in Figure 2. The process to be controlled is assumed to
be of the form
x_ = Ax + Bu + Gv where x is the process state, u the input and v is white noise with unit incremental variance. Discretizing the process in the sampling instants kh taking into account the e ect of the time delays ksc and kca, see Figure 2, gives the discrete time model xk+1 =xk + ?0 ( ksc ; kca)uk + ?1( ksc ; kca)uk?1 + vk (1) yk =Cxk + ek (2) where =eAh ?0( ksc ; kca) = ?1( ksc ; kca) =
Z h? ksc ? kca 0
Zh
h? ksc ? kca
eAs dsB
eAs dsB
and vk and ek are uncorrelated white noise with zero mean and covariance matrices R1 and R2 respectively. Denote the information available when control signal uk is calculated by Yk . This has the structure
Yk = yk ; yk?1; :::; ksc; ksc?1; ::::; kca?1; kca?2; :::; uk?1; uk?2; ::: : Notice that the sensor to controller delay sc at time k and older are available, and that the controller to actuator delays ca are assumed known up to time k ? 1. We assume that the control signal is a function of all information available when it is calculated, i.e. uk = f (Yk ).
3. EVALUATION OF SCHEMES It turns out that with the proposed control schemes the closed loop system can be written as zk+1 = A( k )zk + B ( k )wk where k is a random process uncorrelated with wk , zk is a state vector for the closed loop system and wk is a vector with independent white noise components with unit variance. For example zk can be a vector with xk , uk?1, and the controller state. Similarly k can be a vector consisting of ksc and kca . Often zk has the property that xk and uk can be obtained by a linear transformation of zk , such that xk uk = Q s zk :
One way to compare the performance of di erent control schemes when subjected to random communication delays is to set up a cost function to be minimized by the controller. For the LQG-case it is of interest to evaluate the cost function
xk T xk Q u E ;w uk k = ;w E zkT QTs QQszk = tr QTs QQsTk (3)
where Tk = ;w E zk zkT . To evaluate the quantity Tk , which is independent of Q and Qs , we use that Tk+1 = ;w E [Ak zk + Bk wk ][Akzk + Bk wk ]T = E [Bk BkT + Ew Ak zk zkT ATk ]: Using Kronecker products this can be written as B B T ) + Ek (Ak Ak ) vec(Tk ): (4) vec(Tk+1 ) = vec(E k k k
The short forms Ak = A( k ) and Bk = B ( k ) are used to simplify reading. We have here used that zk and wk are independent and that zk is independent of k . This is crucial for the applied technique to work and indirectly requires that k and k?1 are independent. From the above calculations it follows a direct algorithm to calculate the stationary cost function in the LQGcase. Algorithm 1|Stationary cost function
1. Iterate (4) forward in time to get the stationary value T1 of Tk . 2. Calculate the stationary cost function from (3). Stochastic Stability From (4) it is seen that stochastic stability of the closed loop system is determined by the stability of the matrix (5) E (A( k ) A( k )): k We have now derived formulas for evaluation of cost functions from equations for the corresponding closed loop system. In the case of quadratic cost function an algorithm for evaluation of the mean cost has been found. A condition for test of stochastic stability followed from the calculations. 4. OPTIMAL STOCHASTIC CONTROL In this section we solve the control problem set up by the cost function N x T x X k k (6) JN = E Q uk k=0 uk
The solution of this problem follows by the same technique as for the standard LQG problem. We have the following result: Theorem 1|Optimal state feedback
Given the plant (1), with noise free measurement of the state vector xk , i.e. yk = xk . The control law that minimizes the cost function (6) is given by x k 22 ? 1 T 21 23 ~ ~ ~ uk = ?(Q22 + Sk+1 ) [ Q12 + Sk+1 Sk+1 ] uk?1 (7) where S~k+1 ( sc ) = Eca GT ( sc; ca )Sk+1 G( sc; ca) ?0( sc ; ca) ?1( sc ; ca) G( sc; ca ) = 0 I 0 T sc sc Sk = Esc fF1 ( )QF1( ) + F2T ( sc )S~k+1 ( sc )F2( sc )g # " ~22 ( Q 0 22 + Sk +1 ) sc 22 ? 1 F1( ) =(Q22 + S~k+1) ?(QT12 + S~k21+1) ?S~k23+1 sc F2( sc ) = F1 ( ) [0 I ] SN =0: S~kij is block (i; j ) of S~k ( sc ), and Qij is block (i; j ) of Q. Proof sketch Introduce a new state variable zk = xk uk?1 . Using dynamic programming with Sk the cost to go at time k, and with k the part of the cost function
that cannot be a ected by control, gives
zkT Sk zk + k
(
xk uk
T
)
x Q uk + zkT+1 Sk+1 zk+1 k k k + k+1 ( T xk Q xk E = Esc min ca uk uk k uk k ;vk o T Sk+1 zk+1 sc + k+1 + z k ( T k+1 x x k = Esc min Q k uk uk k uk 39 2 3T 2 xk > xk > = 7 6 7 6 ~ + 4 uk 5 Sk+1 4 uk 5> + tr Sk11+1 R1 + k+1: ; uk?1 > uk?1
E = min uk sc ; ca ;vk
The second equality follows from the fact that ksc is known when uk is determined. The third equality x follows from independence of k and kca , and from uk the de nition of S~k+1 . The resulting expression is a quadratic form in uk . Minimizing this with respect to uk gives the optimal control law (7). Theorem 1 states that the optimal controller with full state information is a linear ksc-depending feedback from the state and the previous control signal, i.e. x k sc uk = ?L( k ; Sk+1) : uk?1 The equation involved in going from Sk+1 to Sk is a stochastic Riccati equation evolving backwards in time. Each step in this iteration will contain expectation calculations with respect to the unknown ksc and kca . Under reasonable assumptions a stationary value S1 of Sk can be found by iterating the stochastic Riccati equation. In practice a tabular for L( ksc ; S1 ) can then be calculated to get a control law on the form xk sc uk = ?L( k ) u k?1 where L( ksc ) is interpolated from the tabular values of L( ksc ; S1 ) in real-time. In many cases the assumption of full state information does not hold. This can be solved by constructing a state estimate from the available data. In our setup there is the problem of the random time delays which enter in a nonlinear fashion. The fact that the old time delays up to time k ? 1 are known at time k, however, allows the standard Kalman lter of the process state to be optimal: Theorem 2|Optimal state estimate
Given the plant (1)-(2). The estimator
x^kjk = x^kjk?1 + K k (yk ? C x^kjk?1) with
x^k+1jk = ^xkjk?1 + ?0( ksc ; kca)uk + ?1 ( ksc ; kca)uk?1 + Kk (yk ? C x^kjk?1) x^0j?1 = E(x0 ) Pk+1 = Pk T + R1 ? Pk C T [CPk C T + R2]?1CPk P0 = R0 = E(x0xT0 ) Kk = Pk C T [CPkC T + R2]?1 K k = Pk C T [CPkC T + R2]?1
(8)
minimizes the error variance Ef[xk ? x^k ]T [xk ? x^k ] j Yk g. The lter gain Kk is independent of sc and ca. Moreover the estimation error is Gaussian with zero mean and covariance Pk ? Pk C T [CPkC T + R2]?1CPk .
Proof This can be proved as in Anderson and Moore (1979).
The following theorem justi es use of the estimated state in the optimal controller. Theorem 3|Separation property
Given the plant (1){(2), with Yk known when the control signal is calculated. The controller that minimizes the cost function (6) is given by x^ uk = ?(Q22 + S~k22+1 )?1 [ QT12 + S~k21+1 S~k23+1 ] kjk uk?1 where S~k is calculated as in Theorem 1, and x^kjk is the minimum variance estimate from Theorem 2. Full detailed proofs of Theorems 1{3 are available in Nilsson (1996). 5. A SUBOPTIMAL SCHEME A drawback with the optimal scheme is the complicated state feedback matrix L( ksc ). An alternative to the optimal controller is the suboptimal controller x ^kjk p p (9) uk = ?L [ k ?k ] uk?1 where pk = e
A( ksc +E kca )
?pk =
Z ksc +E kca 0
eAs dsB
and L is the optimal state feedback vector in the delayfree setup. Ep kca is the mean value of kca . The operation p k x^kjk + ?k uk?1 can be seen as a prediction from the state estimate at time kh to a state estimate when the control signal is applied at the actuator. This controller requires less computations than the optimal controller in Section 4. In Section 6 this controller is compared with the optimal controller. 6. EXAMPLE Consider the following plant, both plant and design speci cations are taken from Doyle and Stein (1979), dx = 0 1 x + 0 u + 35 (10) dt ?61 1 ?3 ?4 y = [2 1] x +
where E[ (t)] = E[(t)] = 0 and E[ (t1 ) (t2 )] = E[(t1)(t2 )] = (t1 ? t2). The control objective is to minimize the cost function 1 Z T (xT H T Hx + u2 )dt J = E Tlim !1 T 0
p p
where H = 4 5 [ 35 1 ]. The sampling period for the controller is chosen as h = 0:05. This is in accordance with the rule of thumb that is given in Åström and Wittenmark (1990). The time delays, ksc and kca , are assumed to be uniformly distributed on the interval [0; h=2] where 0 1. The stationary cost function will be evaluated and compared for four di erent schemes, an LQG-controller neglecting the time delays, the scheme with bu ers proposed in Luck and Ray (1990), the optimal controller derived in Section 4, and the suboptimal controller in Section 5. The rst design is done without taking any time delays into account. The process and the cost function are sampled to get discrete time equivalents, and the standard LQG-controller is calculated. This gives the design :911 T ; K = 2:690 ; K = 2:927 L = 38 ?4:484 8:094 ?5:012
This choice of L, K and K gives the following closed loop poles
sp( ? ?L) = f0:700 0:0702ig sp( ? KC ) = f0:743; 0:173g Even if these looks reasonable a Nyquist plot of the loop transfer function reveals a small phase margin, m = 10:9. The small phase margin indicates that there could be problems to handle unmodeled time delays. Numerical evaluation of (5) gives the stability limit crit = 0:249 for the controller neglecting the time delays. The scheme in Luck and Ray (1990) eliminates the randomness of the time delays by introduction of timed bu ers. This will, however, introduce extra time delay in the loop. The design for this scheme is done in the same way as in the standard LQG-problem. The third scheme we will compare is the optimal controller described in Section 4. Notice that the optimal state estimator gains K and K will be the same for the optimal controller as if the time delays were neglected. The feedback from the estimated state will have the form x^k sc uk = ?L( k ) u k?1
Control strategies when the control delay can be
4
5
x 10
Design neglecting time delays
4.5
larger than the sampling interval. A problem that occurs in this case is that there are then no guarantee that the samples arrive at the controller and the actuator in the order they were sent. A related problem is when samples may be lost in the communication, so called vacant sampling.
4
Scheme by Luck-Ray
3.5 3
Loss
2.5 2
8. REFERENCES
Suboptimal controller Optimal controller
1.5 1 0.5 0 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig. 3. Exact calculated performance (solid lines) of the four schemes, and simulated performance (dashed lines) of system (10). Notice the small di erence between the suboptimal controller and the optimal controller. For > 0:249 the controller neglecting the time delays fails to stabilize the process.
The suboptimal controller (9) uses L, K , and K from the standard LQG-controller. The stationary cost function has been evaluated for the four schemes with Algorithm 1. For comparison the stationary cost has also been evaluated by Monte Carlo simulation, which is made by calculating the mean cost during 4 104 simulated samples. The results agree very well, see Figure 3. From Figure 3 it is seen that the controller neglecting the time delays fails to stabilize the process for > crit. The optimal controller and the proposed suboptimal scheme outperforms the scheme proposed in Luck and Ray (1990). Note that for this example the cost is just slightly higher with the suboptimal controller than with the optimal controller. 7. CONCLUSIONS In this paper we have described a method to analyze performance of di erent schemes to compensate for randomly varying time delays. A condition for test of stochastic stability of the closed loop system has been derived. For the setup with time-driven sampling, event-driven controller, and event-driven actuator, an LQG-optimal controller has been found. The optimal controller has successfully been compared to a proposed suboptimal controller and some controllers proposed in the literature. Future work will include studies of
Optimal schemes when the time delays are correlated from sample to sample.
Anderson, B. and J. Moore (1979): Optimal ltering. Prentice-Hall, Englewood Cli s, N.J. Åström, K. J. and B. Wittenmark (1990): Computer Controlled Systems|Theory and Design. Prentice-Hall, Englewood Cli s, New Jersey, second edition. Doyle, J. C. and G. Stein (1979): \Robustness with observers." IEEE Trans. Automat. Contr., AC-24:4, pp. 607{611. Krtolica, R., U . Özguner, H. Chan, H. Göktas, J. Winkelman, and M. Liubakka (1994): \Stability of linear feedback systems with random communication delays." Int. J. Control, 59:4, pp. 925{953. Luck, R. and A. Ray (1990): \An observer-based compensator for distributed delays." Automatica, 26:5, pp. 903{ 908. Nilsson, J. (1996): Lic Tech thesis, Department of Automatic Control, Lund Institute of Technology, Lund, Sweden. Ray, A. (1989): \Introduction to networking for integrated control systems." IEEE Control Systems Magazine, January, pp. 76{79. Ray, A. (1994): \Output feedback control under randomly varying distributed delays." Journal of Guidance, Control, and Dynamics, 17:4, pp. 701{711. Shin, K. G. and H. Kim (1992): \Hard deadlines in realtime systems." In IFAC Symposium on Algorithms and Architectures for Real-Time Control, pp. 9{14, Seoul, Korea. Voulgaris, P. (1994): \Control of asynchronous sampled data systems." IEEE Trans. Automat. Contr., 39:7, pp. 1451{ 1455. Wittenmark, B., J. Nilsson, and M. Törngren (1995): \Timing problems in real-time control systems." In Preprints American Control Conference, pp. 2000{2004, Seattle, WA.
