A Game Theory Approach to Constrained Minimax ... - Semantic Scholar
A Game Theory Approach to Constrained Minimax State Estimation Dan Simon ¤ Cleveland State University Department of Electrical and Computer Engineering Stilwell Hall Room 332 2121 Euclid Avenue Cleveland, OH 44115 April 6, 2005
Abstract
Sometimes state constaints are enforced heuristically in Kalman ¯lters [11]. Some researchers have treated state constraints by reducing the system model parameterization [27], but this approach is not always desirable or even possible [28]. Other researchers treat state constraints as perfect measurements [7, 16]. This results in a singular covariance matrix but does not present any theoretical problems [4]. In fact, Kalman's original paper [10] presents an example that uses perfect measurements (i.e., no measurement noise). But there are several considerations that indicate against the use of perfect measurements in a Kalman ¯lter implementation. Although the Kalman ¯lter does not formally require a nonsingular covariance matrix, in practice a singular covariance increases the possibility of numerical problems [12, p. 249], [24, p. 365]. Also, the incorporation of state constraints as perfect measurements increases the dimension of the problem, which in turn increases the size of the matrix that needs to be inverted in the Kalman gain computation. These issues are addressed in [19], which develops a constrained Kalman ¯lter by projecting the standard Kalman ¯lter estimate onto the constraint surface. Numerous e®orts have been pursued to incorporate constraints into H1 control problems. For instance, H1 control can be achieved subject to constraints on the system time response [8, 17, 18], state variables [14], controller poles [25], state integrals [13], and control variables [1, 33]. Fewer attempts have been made to incorporate constraints into H1 ¯ltering problems. One example is H1 ¯lter design with poles that are constrained to a speci¯c region [15]. FIR and IIR ¯lters can be designed such that the H1 norm of the error transfer function is minimized while constraining the ¯lter output to lie within a prescribed envelope [26, 32]. However, to our knowledge, there have not been any e®orts to incorporate state equality constraints into H1 ¯ltering problems. This paper generalizes the results of [30] so that minimax state estimation can be performed while satisfying equality constraints on the state estimate. The major
This paper presents a game theory approach to the constrained state estimation of linear discrete time dynamic systems. In the application of state estimators there is often known model or signal information that is either ignored or dealt with heuristically. For example, constraints on the state values (which may be based on physical considerations) are often neglected because they do not easily ¯t into the structure of the state estimator. This paper develops a method for incorporating state equality constraints into a minimax state estimator. The algorithm is demonstrated on a simple vehicle tracking simulation. Key Words { H1 Filter, Minimax Filter, State Constraints, State Estimation, Game Theory.
1
Introduction
In the application of state estimators there is often known model or signal information that is either ignored or dealt with heuristically [11]. This paper presents a way to generalize a minimax state estimator in such a way that known relations among the state variables (i.e., state constraints) are satis¯ed by the state estimate. Constrained state estimation has not, to our knowledge, been studied from a game theory or minimax point of view. Interest in minimax estimation (also called H1 estimation) began in 1981 [31], when it was noted that in dealing with noise with unknown statistics, the noise could be modeled as a deterministic signal. This replaces the Kalman ¯ltering method of modeling the noise as a random process. This results in estimators that are more robust to unmodeled noise and uncertainty, as will be illustrated in Section 5. Although state constraints have not yet been incorporated into minimax ¯lters, they have been incorporated into Kalman ¯lters using a variety of di®erent approaches. ¤ Email: [email protected], Phone: 216-687-5407, Fax: 216-687-5405
1
of physical considerations or other a priori information) that the states satisfy the following constraint.
contribution of this paper is the development of a minimax state estimator for linear systems that enforces equality constraints on the state estimate. We formulate the problem as a particular game which was shown in [30] to be equivalent to an H1 state estimation problem. We then derive the estimator gain and adversary gain that yields a saddle point for the constrained estimation problem. Constrained estimators other than H1 ¯lters can be implemented on constrained problems. The most notable alternative to constrained H1 ¯ltering is constrained Kalman ¯ltering [19]. The choice of whether to use a constrained Kalman or constrained H1 ¯lter is problem dependent, but the general advantages of H1 estimation can be summarized as follows [6].
Dk xk = dk
We assume that the Dk matrix is full rank and normalized so that Dk DkT = I. In general, Dk is an s £ n matrix, where s is the number of constraints, n is the number of states, and s < n. If s = n then (2) completely de¯nes xk , which makes the estimation problem trivial. For s < n, which is the case in this paper, there are fewer constraints than states, which makes the estimation problem nontrivial. Assuming that Dk is full rank is the same as the assumption made in the constrained Kalman ¯ltering problem [19]. We de¯ne the following matrix for notational convenience.
² H1 ¯ltering provides a rigorous method for dealing with systems that have model uncertainty.
Vk = DkT Dk
² Continuous time H1 ¯ltering provides a natural way to limit the frequency response of the estimator. (Although this paper deals strictly with discrete time ¯ltering, the methods herein can also be used to extend existing continuous time H1 ¯ltering results to constrained ¯ltering. This is an area for further research.)
We will assume that both the noisy system and the noisefree system satisfy the above state constraint. The problem is to ¯nd an estimate x ^k+1 of xk+1 given the measurements fy0 ; y1 ; ¢ ¢ ¢ ; yk g. The estimate should satisfy the state constraint. We will restrict the state estimator to have an observer structure so that it results in an unbiased estimate [2].
² H1 ¯ltering can be used to guarantee stability margins or minimize worst case estimation error. ² H1 ¯ltering may more appropriate for systems where the model changes unpredictably, and model identi¯cation and gain scheduling are too complex or time consuming.
Axk + Bwk + ±k
yk
=
Cxk + mk
0
x ^k+1
=
A^ xk + Kk (yk ¡ C x ^k )
~k Kk = (I ¡ Vk+1 )K
(4)
(5)
~ k is any dimensionally appropriate matrix, then where K the state estimate (4) satis¯es the state constraint (2). Proof: See the appendix. The noise ±k in (1) is introduced by an adversary that has the goal of maximizing the estimation error. Similar to [30], we will assume that our adversary's input to the system is given by
Consider the discrete linear time-invariant system given by =
=
Lemma 1 If we have an estimator gain of the form
Problem Statement
xk+1
x ^0
(3)
The main advantage of unbiased estimators over biased estimators is that unbiased estimators make it easier to quantify the estimation error. With biased estimators we must quantify the error using both the bias and some other measure (e.g., mean square error or worst case error). In general, unbiased estimators are preferred over biased estimators because of their greater mathematical tractability.
Section 2 of this paper formulates the problem, and Section 3 develops the solution through a series of preliminary lemmas and the main saddle point theorem of this paper. As expected, it turns out that the unconstrained minimax estimator is a special case of the constrained minimax estimator. Section 4 discuss how the methods of this paper can be extended to inequality constraints. Section 5 presents some simulation results, and Section 6 o®ers some concluding remarks. Some lemma proofs are presented in the appendix.
2
(2)
(1)
±k = Lk (Gk (xk ¡ x ^k ) + nk )
(6)
where Lk is a gain to be determined, Gk is a given matrix, and fnk g is a noise sequence. We will assume that fwk g, fmk g, and fnk g are mutually uncorrelated unityvariance white noise sequences that are uncorrelated with x0 . This form of the adversary's input is not intuitive because it uses the state estimation error, but this form is taken because the solution of the resulting problem results in a state estimator that bounds the in¯nity norm
where k is the time index, x is the state vector, y is the measurement, fwk g and fmk g are white noise sequences, and f±k g is a noise sequence generated by an adversary. We assume that fwk g and fmk g are mutually uncorrelated unity-variance white noise sequences. In general, A, B, and C can be time-varying matrices, but we will omit the time subscript on these matrices for ease of notation. In addition to the state equation we know (on the basis
2
of the transfer function from the random noise terms to the state estimation error [30]. (This is discussed further following (15).) Gk can be considered by the designer as a tuning parameter or weighting matrix that can be adjusted on the basis of our a priori knowledge about the adversary's noise input. Suppose, for example, that we know ahead of time that the ¯rst component of the adversary's noise input to the system is twice the magnitude of the second component, the third component is zero, etc.; then that information can be re°ected in the designer's choice of Gk . We do not need to make any assumptions about the form of Gk (e.g., it does not need to be positive de¯nite or square). From (6) we can see that as Gk approaches the zero matrix, the adversary's input becomes purely a random process without any deterministic component. This causes the optimal minimax ¯lter to approach the Kalman ¯lter; that is, we obtain better RMS error performance but not as good worst-case error performance. As Gk becomes large, the minimax ¯lter places more emphasis on minimizing the estimation error due to the deterministic component of the adversary's input. That is, the minimax ¯lter assumes less about the adversary's input, and we obtain better worst-case error performance but worse RMS error performance.
However, this is an inappropriate term for a minimax problem because the adversary can arbitrarily increase ek by arbitrarily increasing Lk . To prevent this, we decompose ek as follows. ek = e1;k + e2;k where e1;k and e2;k evolve as follows. e1;0
=
x0
e1;k
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk)e1;k +
e2;0
=
0
e2;k
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk )e2;k + Lk nk
(13)
Bwk ¡ Kk mk
(14)
Motivated by [30], we de¯ne the objective function as J(K; L) = trace
N X k=0
Wk E(e1;k eT1;k ¡ e2;k eT2;k )
(15)
where Wk is any positive de¯nite weighting matrix. The di®erential game is for the ¯lter designer to ¯nd a gain sequence fKk g that minimizes J, and for the adversary to ¯nd a gain sequence fLk g that maximizes J. As such, J is considered a function of fKk g and fLk g, which we denote in shorthand notation as K and L. This objective function is not intuitive, but is used here because the solution of the problem results in a state estimator that bounds the in¯nity norm of the transfer function from the random noise terms to the state estimation error [30]. This is expressed more completely in the next section in Lemma 3.
Lemma 2 In order for the noise-free system (1) to satisfy the state constraint (2), the adversary gain Lk must satisfy the following equality. Dk+1 Lk = 0
(12)
(7)
One way to satisfy this equality is for Lk to be of the form ~k Lk = (I ¡ Vk+1 )L
3
(8)
The solution is obtained by ¯nding optimal gain sequences fKk¤ g and fL¤k g that satisfy the following saddle point.
~ k is any dimensionally appropriate matrix. where L
J(K ¤ ; L) · J(K ¤ ; L¤ ) · J(K; L¤ ) for all K; L
Proof: See the appendix. The estimation error is de¯ned as follows. ek = xk ¡ x ^k
=
x0
ek+1
=
(A ¡ Kk C + Lk Gk )ek +
(9)
De¯ne the following matrix di®erence equation.
(10)
=
x0
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk )ek +
=
E(x0 xT0 )
Qk+1
=
Fk Qk FkT + BB T + Kk KkT ¡ Lk LTk
(18)
Lemma 3 The cost function (15) is given as follows.
Since Dk xk = Dk x ^k = dk , we see that Dk ek = 0. But we also know by following a procedure similar to the proof of Lemma 1 that Dk+1 Aek = 0. Therefore we can subtract T Dk+1 Aek = Vk+1 Aek from the error the zero term Dk+1 equation (10) to obtain the following. e0
Q0
Then we have the following lemma.
Bwk + Lk nk ¡ Kk mk
ek+1
(16)
To solve this problem we will write the cost function (15) in a more convenient form. De¯ne the matrix Fk as follows. (17) Fk = (I ¡ Vk+1 )A ¡ Kk C + Lk Gk
It can be shown from the preceding equations that the dynamic system describing the evolution of the estimation error is given as follows. e0
Problem Solution
J(K; L) = trace
N X
Wk Qk
(19)
k=0
Also, the minimization of J(K; L¤ ) with respect to K results in an estimator with the following bound for the square of the induced l2 norm of the system.
(11)
sup PN
Bwk + Lk nk ¡ Kk mk
wk ;mk
3
PN
k=0
k=0
jjGk ek jj22
(jjwk jj22 + jjmk jj22 )
Abstract
Sometimes state constaints are enforced heuristically in Kalman ¯lters [11]. Some researchers have treated state constraints by reducing the system model parameterization [27], but this approach is not always desirable or even possible [28]. Other researchers treat state constraints as perfect measurements [7, 16]. This results in a singular covariance matrix but does not present any theoretical problems [4]. In fact, Kalman's original paper [10] presents an example that uses perfect measurements (i.e., no measurement noise). But there are several considerations that indicate against the use of perfect measurements in a Kalman ¯lter implementation. Although the Kalman ¯lter does not formally require a nonsingular covariance matrix, in practice a singular covariance increases the possibility of numerical problems [12, p. 249], [24, p. 365]. Also, the incorporation of state constraints as perfect measurements increases the dimension of the problem, which in turn increases the size of the matrix that needs to be inverted in the Kalman gain computation. These issues are addressed in [19], which develops a constrained Kalman ¯lter by projecting the standard Kalman ¯lter estimate onto the constraint surface. Numerous e®orts have been pursued to incorporate constraints into H1 control problems. For instance, H1 control can be achieved subject to constraints on the system time response [8, 17, 18], state variables [14], controller poles [25], state integrals [13], and control variables [1, 33]. Fewer attempts have been made to incorporate constraints into H1 ¯ltering problems. One example is H1 ¯lter design with poles that are constrained to a speci¯c region [15]. FIR and IIR ¯lters can be designed such that the H1 norm of the error transfer function is minimized while constraining the ¯lter output to lie within a prescribed envelope [26, 32]. However, to our knowledge, there have not been any e®orts to incorporate state equality constraints into H1 ¯ltering problems. This paper generalizes the results of [30] so that minimax state estimation can be performed while satisfying equality constraints on the state estimate. The major
This paper presents a game theory approach to the constrained state estimation of linear discrete time dynamic systems. In the application of state estimators there is often known model or signal information that is either ignored or dealt with heuristically. For example, constraints on the state values (which may be based on physical considerations) are often neglected because they do not easily ¯t into the structure of the state estimator. This paper develops a method for incorporating state equality constraints into a minimax state estimator. The algorithm is demonstrated on a simple vehicle tracking simulation. Key Words { H1 Filter, Minimax Filter, State Constraints, State Estimation, Game Theory.
1
Introduction
In the application of state estimators there is often known model or signal information that is either ignored or dealt with heuristically [11]. This paper presents a way to generalize a minimax state estimator in such a way that known relations among the state variables (i.e., state constraints) are satis¯ed by the state estimate. Constrained state estimation has not, to our knowledge, been studied from a game theory or minimax point of view. Interest in minimax estimation (also called H1 estimation) began in 1981 [31], when it was noted that in dealing with noise with unknown statistics, the noise could be modeled as a deterministic signal. This replaces the Kalman ¯ltering method of modeling the noise as a random process. This results in estimators that are more robust to unmodeled noise and uncertainty, as will be illustrated in Section 5. Although state constraints have not yet been incorporated into minimax ¯lters, they have been incorporated into Kalman ¯lters using a variety of di®erent approaches. ¤ Email: [email protected], Phone: 216-687-5407, Fax: 216-687-5405
1
of physical considerations or other a priori information) that the states satisfy the following constraint.
contribution of this paper is the development of a minimax state estimator for linear systems that enforces equality constraints on the state estimate. We formulate the problem as a particular game which was shown in [30] to be equivalent to an H1 state estimation problem. We then derive the estimator gain and adversary gain that yields a saddle point for the constrained estimation problem. Constrained estimators other than H1 ¯lters can be implemented on constrained problems. The most notable alternative to constrained H1 ¯ltering is constrained Kalman ¯ltering [19]. The choice of whether to use a constrained Kalman or constrained H1 ¯lter is problem dependent, but the general advantages of H1 estimation can be summarized as follows [6].
Dk xk = dk
We assume that the Dk matrix is full rank and normalized so that Dk DkT = I. In general, Dk is an s £ n matrix, where s is the number of constraints, n is the number of states, and s < n. If s = n then (2) completely de¯nes xk , which makes the estimation problem trivial. For s < n, which is the case in this paper, there are fewer constraints than states, which makes the estimation problem nontrivial. Assuming that Dk is full rank is the same as the assumption made in the constrained Kalman ¯ltering problem [19]. We de¯ne the following matrix for notational convenience.
² H1 ¯ltering provides a rigorous method for dealing with systems that have model uncertainty.
Vk = DkT Dk
² Continuous time H1 ¯ltering provides a natural way to limit the frequency response of the estimator. (Although this paper deals strictly with discrete time ¯ltering, the methods herein can also be used to extend existing continuous time H1 ¯ltering results to constrained ¯ltering. This is an area for further research.)
We will assume that both the noisy system and the noisefree system satisfy the above state constraint. The problem is to ¯nd an estimate x ^k+1 of xk+1 given the measurements fy0 ; y1 ; ¢ ¢ ¢ ; yk g. The estimate should satisfy the state constraint. We will restrict the state estimator to have an observer structure so that it results in an unbiased estimate [2].
² H1 ¯ltering can be used to guarantee stability margins or minimize worst case estimation error. ² H1 ¯ltering may more appropriate for systems where the model changes unpredictably, and model identi¯cation and gain scheduling are too complex or time consuming.
Axk + Bwk + ±k
yk
=
Cxk + mk
0
x ^k+1
=
A^ xk + Kk (yk ¡ C x ^k )
~k Kk = (I ¡ Vk+1 )K
(4)
(5)
~ k is any dimensionally appropriate matrix, then where K the state estimate (4) satis¯es the state constraint (2). Proof: See the appendix. The noise ±k in (1) is introduced by an adversary that has the goal of maximizing the estimation error. Similar to [30], we will assume that our adversary's input to the system is given by
Consider the discrete linear time-invariant system given by =
=
Lemma 1 If we have an estimator gain of the form
Problem Statement
xk+1
x ^0
(3)
The main advantage of unbiased estimators over biased estimators is that unbiased estimators make it easier to quantify the estimation error. With biased estimators we must quantify the error using both the bias and some other measure (e.g., mean square error or worst case error). In general, unbiased estimators are preferred over biased estimators because of their greater mathematical tractability.
Section 2 of this paper formulates the problem, and Section 3 develops the solution through a series of preliminary lemmas and the main saddle point theorem of this paper. As expected, it turns out that the unconstrained minimax estimator is a special case of the constrained minimax estimator. Section 4 discuss how the methods of this paper can be extended to inequality constraints. Section 5 presents some simulation results, and Section 6 o®ers some concluding remarks. Some lemma proofs are presented in the appendix.
2
(2)
(1)
±k = Lk (Gk (xk ¡ x ^k ) + nk )
(6)
where Lk is a gain to be determined, Gk is a given matrix, and fnk g is a noise sequence. We will assume that fwk g, fmk g, and fnk g are mutually uncorrelated unityvariance white noise sequences that are uncorrelated with x0 . This form of the adversary's input is not intuitive because it uses the state estimation error, but this form is taken because the solution of the resulting problem results in a state estimator that bounds the in¯nity norm
where k is the time index, x is the state vector, y is the measurement, fwk g and fmk g are white noise sequences, and f±k g is a noise sequence generated by an adversary. We assume that fwk g and fmk g are mutually uncorrelated unity-variance white noise sequences. In general, A, B, and C can be time-varying matrices, but we will omit the time subscript on these matrices for ease of notation. In addition to the state equation we know (on the basis
2
of the transfer function from the random noise terms to the state estimation error [30]. (This is discussed further following (15).) Gk can be considered by the designer as a tuning parameter or weighting matrix that can be adjusted on the basis of our a priori knowledge about the adversary's noise input. Suppose, for example, that we know ahead of time that the ¯rst component of the adversary's noise input to the system is twice the magnitude of the second component, the third component is zero, etc.; then that information can be re°ected in the designer's choice of Gk . We do not need to make any assumptions about the form of Gk (e.g., it does not need to be positive de¯nite or square). From (6) we can see that as Gk approaches the zero matrix, the adversary's input becomes purely a random process without any deterministic component. This causes the optimal minimax ¯lter to approach the Kalman ¯lter; that is, we obtain better RMS error performance but not as good worst-case error performance. As Gk becomes large, the minimax ¯lter places more emphasis on minimizing the estimation error due to the deterministic component of the adversary's input. That is, the minimax ¯lter assumes less about the adversary's input, and we obtain better worst-case error performance but worse RMS error performance.
However, this is an inappropriate term for a minimax problem because the adversary can arbitrarily increase ek by arbitrarily increasing Lk . To prevent this, we decompose ek as follows. ek = e1;k + e2;k where e1;k and e2;k evolve as follows. e1;0
=
x0
e1;k
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk)e1;k +
e2;0
=
0
e2;k
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk )e2;k + Lk nk
(13)
Bwk ¡ Kk mk
(14)
Motivated by [30], we de¯ne the objective function as J(K; L) = trace
N X k=0
Wk E(e1;k eT1;k ¡ e2;k eT2;k )
(15)
where Wk is any positive de¯nite weighting matrix. The di®erential game is for the ¯lter designer to ¯nd a gain sequence fKk g that minimizes J, and for the adversary to ¯nd a gain sequence fLk g that maximizes J. As such, J is considered a function of fKk g and fLk g, which we denote in shorthand notation as K and L. This objective function is not intuitive, but is used here because the solution of the problem results in a state estimator that bounds the in¯nity norm of the transfer function from the random noise terms to the state estimation error [30]. This is expressed more completely in the next section in Lemma 3.
Lemma 2 In order for the noise-free system (1) to satisfy the state constraint (2), the adversary gain Lk must satisfy the following equality. Dk+1 Lk = 0
(12)
(7)
One way to satisfy this equality is for Lk to be of the form ~k Lk = (I ¡ Vk+1 )L
3
(8)
The solution is obtained by ¯nding optimal gain sequences fKk¤ g and fL¤k g that satisfy the following saddle point.
~ k is any dimensionally appropriate matrix. where L
J(K ¤ ; L) · J(K ¤ ; L¤ ) · J(K; L¤ ) for all K; L
Proof: See the appendix. The estimation error is de¯ned as follows. ek = xk ¡ x ^k
=
x0
ek+1
=
(A ¡ Kk C + Lk Gk )ek +
(9)
De¯ne the following matrix di®erence equation.
(10)
=
x0
=
((I ¡ Vk+1 )A ¡ Kk C + Lk Gk )ek +
=
E(x0 xT0 )
Qk+1
=
Fk Qk FkT + BB T + Kk KkT ¡ Lk LTk
(18)
Lemma 3 The cost function (15) is given as follows.
Since Dk xk = Dk x ^k = dk , we see that Dk ek = 0. But we also know by following a procedure similar to the proof of Lemma 1 that Dk+1 Aek = 0. Therefore we can subtract T Dk+1 Aek = Vk+1 Aek from the error the zero term Dk+1 equation (10) to obtain the following. e0
Q0
Then we have the following lemma.
Bwk + Lk nk ¡ Kk mk
ek+1
(16)
To solve this problem we will write the cost function (15) in a more convenient form. De¯ne the matrix Fk as follows. (17) Fk = (I ¡ Vk+1 )A ¡ Kk C + Lk Gk
It can be shown from the preceding equations that the dynamic system describing the evolution of the estimation error is given as follows. e0
Problem Solution
J(K; L) = trace
N X
Wk Qk
(19)
k=0
Also, the minimization of J(K; L¤ ) with respect to K results in an estimator with the following bound for the square of the induced l2 norm of the system.
(11)
sup PN
Bwk + Lk nk ¡ Kk mk
wk ;mk
3
PN
k=0
k=0
jjGk ek jj22
(jjwk jj22 + jjmk jj22 )
