A Geometric Slicing Lower Bound for Average-Cost Dynamic ...
52nd IEEE Conference on Decision and Control December 10-13, 2013. Florence, Italy
A Geometric Slicing Lower Bound for Average-Cost Dynamic Programming Se Yong Park and Anant Sahai
{separk,sahai}@eecs.berkeley.edu
Abstract— A geometric slicing idea is proposed to lower bound infinite-horizon average-cost dynamic programs. The idea divides an infinite-horizon problem into finite-horizon ones with discounted cost. The idea is applied to control-overcommunication-channel problems to find a fundamental limit of such systems. Lower bounds on the performance are given in terms of the capacity of the channel. The lower bounds are compared with explicit control strategies to provide quantitative and qualitative understanding about the strategies.
the simplest LQG (linear quadratic Gaussian) decentralized control problem —a scalar plant with two controllers—, and discovered an approximately optimal strategy with a provable performance guarantee. Precisely, we considered the following distributed system: x[n + 1] = ax[n] + b1 u1 [n] + b2 u2 [n] + w[n] y1 [n] = c1 x[n] + v1 [n] y2 [n] = c2 x[n] + v2 [n]
I. I NTRODUCTION Dynamic programming [1] has provided a unified theoretical framework for control problems involving dynamic systems. Especially in infinite-horizon problems, the whole system converges to a “steady state” behavior, and the solutions of infinite-horizon dynamic programs (in both average-cost and discounted-cost cases) manifest themselves as solutions to the celebrated Hamilton-Jacobi-Bellman (HJB) equations. There are several known numerical algorithms for solving HJB equations [1]. However, it is still extremely difficult to find deeper analytic solutions for general HJB equations, which would provide deeper insight into the problem. Moreover, in lots of control problems, the state and strategy spaces are infinite (for example, in linear systems the states are real vectors). For such cases, the convergence speed of numerical algorithms can be really slow, and even the convergence of the algorithms or the convergence to the optimal is sometimes hard to justify. In distributed control problems where each controller has different information, the situation gets worse. Until recently, a proper way of writing HJB equations for decentralized control problems [2] had been unknown. Even after we have HJB equations, solving the equations is extremely difficult. Combinatorial formulations of distributed control problems are known to be NP-hard. Even for linear systems, the optimal strategy is non-linear (unlike central control of linear systems). Thus, to find the optimal strategy, we have to search over an infinite-dimensional feasible strategy space. Moreover, the optimization problem is known to be nonconvex. Thus, optimal decentralized control problems have been wide open for decades. We refer to [3] for a review of the state of art. Recently, the authors made progress in attacking infinitehorizon decentralized control problems. In [4], we considered This work was supported by the National Science Foundation (CCF729122). Authors are with the Department of Electrical Engineering and Computer Sciences at the University of California at Berkeley.
978-1-4673-5716-6/13/$31.00 ©2013 IEEE
(1)
2 ), v1 [n] ∼ where x[0] ∼ N (0, σ02 ), w[n] ∼ N (0, σw 2 2 N (0, σv1 ), v2 [n] ∼ N (0, σv2 ) are independent. The control objective is minimizing the following quadratic cost:
inf lim sup
u1 ,u2 N →∞
1 X qE[x2 [n]] + r1 E[u21 [n]] + r2 E[u22 [n]]. N 0≤i
A Geometric Slicing Lower Bound for Average-Cost Dynamic Programming Se Yong Park and Anant Sahai
{separk,sahai}@eecs.berkeley.edu
Abstract— A geometric slicing idea is proposed to lower bound infinite-horizon average-cost dynamic programs. The idea divides an infinite-horizon problem into finite-horizon ones with discounted cost. The idea is applied to control-overcommunication-channel problems to find a fundamental limit of such systems. Lower bounds on the performance are given in terms of the capacity of the channel. The lower bounds are compared with explicit control strategies to provide quantitative and qualitative understanding about the strategies.
the simplest LQG (linear quadratic Gaussian) decentralized control problem —a scalar plant with two controllers—, and discovered an approximately optimal strategy with a provable performance guarantee. Precisely, we considered the following distributed system: x[n + 1] = ax[n] + b1 u1 [n] + b2 u2 [n] + w[n] y1 [n] = c1 x[n] + v1 [n] y2 [n] = c2 x[n] + v2 [n]
I. I NTRODUCTION Dynamic programming [1] has provided a unified theoretical framework for control problems involving dynamic systems. Especially in infinite-horizon problems, the whole system converges to a “steady state” behavior, and the solutions of infinite-horizon dynamic programs (in both average-cost and discounted-cost cases) manifest themselves as solutions to the celebrated Hamilton-Jacobi-Bellman (HJB) equations. There are several known numerical algorithms for solving HJB equations [1]. However, it is still extremely difficult to find deeper analytic solutions for general HJB equations, which would provide deeper insight into the problem. Moreover, in lots of control problems, the state and strategy spaces are infinite (for example, in linear systems the states are real vectors). For such cases, the convergence speed of numerical algorithms can be really slow, and even the convergence of the algorithms or the convergence to the optimal is sometimes hard to justify. In distributed control problems where each controller has different information, the situation gets worse. Until recently, a proper way of writing HJB equations for decentralized control problems [2] had been unknown. Even after we have HJB equations, solving the equations is extremely difficult. Combinatorial formulations of distributed control problems are known to be NP-hard. Even for linear systems, the optimal strategy is non-linear (unlike central control of linear systems). Thus, to find the optimal strategy, we have to search over an infinite-dimensional feasible strategy space. Moreover, the optimization problem is known to be nonconvex. Thus, optimal decentralized control problems have been wide open for decades. We refer to [3] for a review of the state of art. Recently, the authors made progress in attacking infinitehorizon decentralized control problems. In [4], we considered This work was supported by the National Science Foundation (CCF729122). Authors are with the Department of Electrical Engineering and Computer Sciences at the University of California at Berkeley.
978-1-4673-5716-6/13/$31.00 ©2013 IEEE
(1)
2 ), v1 [n] ∼ where x[0] ∼ N (0, σ02 ), w[n] ∼ N (0, σw 2 2 N (0, σv1 ), v2 [n] ∼ N (0, σv2 ) are independent. The control objective is minimizing the following quadratic cost:
inf lim sup
u1 ,u2 N →∞
1 X qE[x2 [n]] + r1 E[u21 [n]] + r2 E[u22 [n]]. N 0≤i
