Autonomous Demand Response Using Stochastic Differential Games
Autonomous Demand Response Using Stochastic Differential Games Najmeh Forouzandehmehr∗ , Mohammad Esmalifalak∗ , Hamed Mohsenian-Rad+ , and Zhu Han∗ ∗
Electrical and Computer Engineering Department, University of Houston, Houston, TX, USA + Department of Electrical Engineering, University of California, Riverside, CA, USA
I. I NTRODUCTION
Abstract—Demand Response (DR) programs are implemented to encourage consumers to reduce their electricity demand when needed, e.g., at peak-load hours, by adjusting their controllable load. In this paper, our focus is on controllable load types that are associated with dynamic systems and can be modeled using differential equations. Examples of such load types include heating, ventilation, and air conditioning (HVAC), water heating, and refrigeration. In this regard, we propose a new demand response model based on a two-level differential game framework. At the beginning of each demand response interval, the price is decided by the upper level (aggregator, utility, or market) given the total demand of users in the lower level. At the lower level, for each player (residential or commercial buildings that are equipped with automated load control systems and local renewable generators), given the price from the upper level, the electricity usage of air conditioning unit and the battery storage charging/discharging schedules are controlled in order to minimize the user’s total electricity cost. The optimal user strategies are derived using the stochastic Hamilton-JacobiBellman equations. We also show that the proposed game can converge to a feedback Nash equilibrium. Based on the effect of real-time pricing on users’ daily demand profile, the simulation results demonstrate the properties of the proposed game and show how we can optimize consumers’ electricity cost in presence of time-varying prices.
Demand Response (DR) programs are implemented by utility companies to control the energy consumption at the customer side of the meter. Two popular DR approaches are direct load control (DLC) and smart pricing. In DLC [1]–[4], an aggregator can remotely control the operations and energy consumption of certain consumer appliances. In contrast, in smart pricing, users are encouraged to individually and voluntarily manage their load, e.g., by reducing their consumption at peak price hours. This can be done using automated Energy Consumption Scheduling (ECS) devices [5]. For each user, the ECS finds the best load schedule to minimize the user’s electricity cost while fulfilling the user’s energy needs. This can lead to autonomous demand response programs that burden a minimal control overhead on utilities. A common analytical tool to study autonomous DR systems is game theory [6], that provides a framework to study rational interactions and outcome in a distributed manner. In [7], a stochastic game is developed to model an hourly energy auction in which generators and consumers participate as adaptive agents. In [8], authors proposed a game theoretic demand response scheme to develop a distributed load prediction system that involves user participation. Authors in [9] employed the Cournot game model to analyze the market effect of a demand response aggregator on both shifting and reducing deferrable loads. Authors in [10] developed a hybrid day-ahead and real-time consumption scheduling for a number of houses that participate in a demand side program based on game theory. The interaction between the service provider and the users is modeled as a Stackelberg game in [11] to derive the optimal real-time electricity price and each user’s optimal power consumption. In [12], the real-time pricebased demand response (DR) management is evaluated for residential appliances via stochastic optimization and robust optimization approaches. The work in [12] considers considers uncertainties in real-time electricity prices and determines the optimal operation of residential appliances within 5-minute time. In [13], a dynamic Demand Response (DR) and Distributed Generation (DG) management approach is proposed in the context of smart microgrid for a residential community. The DG management coordinates with DR and considers stochastic elements, such as stochastic load and wind power, to reduce overall energy consumption cost. In [14], a residential energy consumption scheduling framework is proposed, which
Keywords: Stochastic differential game, autonomous demand response, smart building, real-time pricing.
u1 u2 x1 x2 xdi w e l β γ tOD p d g µ L h v N
N OMENCLATURE Power draw from battery for building usage. Air conditioner usage of electricity. The energy stored in the battery array. The indoor temperature of building. The desired temperature. The renewable output prediction. The prediction error. The uncontrollable load of a building. Battery leakage rate. The factor of inertia. The coefficient of performance. The outside temperature. Market spot price. Total power consumption of the building. Total power generation of the building. Building utility function. Building expected total utility function. Terminal condition. Building value function. Number of buildings. 1
attempts to achieve a desired trade-off between minimizing the electricity payment and minimizing the waiting time for the operation of household appliance in presence of real-time prices using price prediction. In [15], game theory is used for demand-side management to reduce the peak-to-average ratio in aggregate load demand. In [16], a tutorial is given for the game-theoretic methods on microgrid systems, demand-side management, and smart grid communications. Different from the prior work in [7]- [16], in this paper, we focus on game-theoretic analysis of price-based DR programs where controllable load types are associated with dynamic systems and can be modeled using differential equations. Examples of such loads include heating, ventilation, and air conditioning (HVAC), water heating, and refrigeration. In particular, we apply techniques from stochastic differential games [17]. To the best of our knowledge, this paper is the first work to study dynamic loads, such as HVAC systems, with differential equations in the context of a game-theoretic demand side management framework. The major efforts and results in this paper can be summarized as follows: 1) We study the strategic interactions between a Nash Cournot electricity market and multiple energy-smart buildings to construct a two-level stochastic differential game framework. At the upper level, the market offers a vector of hourly prices to end users. At the lower level, the energy-smart buildings as the lower level participate in demand response by managing controllable dynamic load in response to hourly prices set by the market. 2) We focus on smart buildings equipped with renewable resources generators, local energy storage and controllable HVAC units, in which users are able to respond to real-time grid conditions like electricity prices and weather conditions in order to minimize their cost. 3) We derive the optimal closed-form control strategies for each energy-smart building obtained by solving stochastic the Hamilton-Jacobi-Bellman (HJB) equation. We analyze the outcome of interactions between two levels and constitute a feedback Nash equilibrium solution. 4) Comparing to day-ahead pricing, the proposed technique makes the load profile more flat and reduces the peakto-average ratio (PAR) of aggregate load. 5) Using simulation results we show that by implementing our proposed stochastic differential DR game model, we can minimize the electricity cost of buildings. The rest of this paper is organized as follows. The system model is described in Section II. The stochastic differential game is constructed and it solution to the proposed game is derived in Section III. Simulation results are presented in Section IV. Conclusions are drawn in Section V.
Fig. 1. The interactions between the aggergator and individual buildings.
the market decides on a price to pass on to the end-users in the lower level, based on the total demand data from the lower level during the last time interval. The ECS unit of each building minimizes the cost of electricity consumption. Since there are multiple buildings competing for the electricity resources, the system can be analyzed using game theory [6]. A. Lower Level (Smart Building Consumers) Consider a total of N energy-smart buildings that participate in demand response program. Each building has two specific controllable loads: an air conditioner with a controllable thermostat, and an always-connected battery. We also assume that a renewable source of energy, e.g., a residential wind turbine or a roof-top solar panel, is available in each building, with its generated output to be used to charge the battery. Uncontrollable appliances with a total and known consumption of l(t) constitute the rest of the building power consumption. Given the price that is a function of the optimal strategy of the upper level player (see Section II-B), the decision variables available for consumers at each building i = 1, . . . , N are: ui1
=
power draw from battery for home usage,
ui2
=
air conditioner usage of electricity,
and the dynamic states include: xi1
= the energy stored in the battery array,
xi2
= the indoor temperature of the home.
The output power of the renewable generator is random and in our analysis, it is modeled as W + ei , where W denotes the renewable output prediction that is obtained using a day-ahead forecasting method and ei denotes the prediction error which is a Gaussian random variable with zero mean and variance σ 2 . As an example, the amount of power generated by a wind turbine can be modeled as a function of wind speed. As for the outside temperature, we assume that its day-ahead predictions are used based on standard weather forecasting data.
II. S YSTEM M ODEL In this section, we explain the system model that incorporates the impact of demand response on both supply and demand sides when real-time pricing is used. As illustrated in Fig. 1, we study a two-level design framework: at the upper level, at the beginning of each time interval, e.g., at each hour, 2
buildings’ power generation and consumption on spot price, p, as follows: PN PN [ j=1 dj (t) − j=1 g j (t)] p(t) = pc (t) + , (2) α where pc (t) is a constant price factor decided by market hourly, dj and g j are the electricity consumption and generation of building j respectively and α is a scalar parameter. Increasing α reduces the impact of buildings on spot price. From Section II-A, for each smart building i, the total power consumption can be calculated as
For each smart building, the dynamics of the states can be modeled using the following differential equations: " #" # " # x˙i1 −β i 0 xi1 = + (1) 0 (i − 1) xi2 x˙i2 " #" # −1 0 ui1 + 0 −γ i (1 − i )K i ui2 " # " # ei 1 w+ . (1 − i )tiOD 0 The differential equation in the first row in (1) models the dynamic of the battery’s state-of-charge. The differential equation in the second row models the variation in the building’s indoor temperature. Here, β i denotes the battery leakage rate. As the battery dynamic equation shows the output of renewable resource W +ei acts as the input to the battery, and the amount of power that is discharged for usage in the building acts as the output of battery. The thermal model in (1) is based on the a building thermal model in [18]. Here, i is the factor of inertia of the building which is a function of time constant of the building and overall thermal conductivity, γ i is the coefficient of performance of the air conditioning unit to cool the air inside the building, tiOD is the outside temperature and K i is a constant that is depends on the performance of the air conditioning unit and the total thermal mass. The air conditioning unit uses power ui2 to cool down the home’s indoor temperature. Note that, in this model, our focus is only on the cooling scenario. The results for the heating scenario are similar and can be obtained by changing the sign of −γ i (1 − i )K i from negative to positive. Notice that for the other deferrable appliances such as dish washers, there is no dynamic model like the air temperature and battery. In other word, if the appliances have done the work, the future will not be affected. This is the assumption in most of existing demand side management literature. For such appliances, it is easy to extend our model in (1) by setting the first term of right hand side with zero memory. Since the major contribution for this paper is to provide solution for the appliances with dynamic models, we do not consider the deferrable appliances without memory in the rest of this paper.
di (t) = li (t) − ui1 (t) + ui2 (t).
(3)
where li (t) is the power consumption beyond HVAC and battery. di (t) can be obtained by smart meter, and ui1 (t) & ui2 (t) are known. So li (t) can be easily calculated and no future value is needed. Another physical interpretation of p(t) can be viewed as the small pertubation around price pc (t). In other word, it is Tylor first order expansion for power market pricing. Since each building’s power consumption is relatively small compared with the whole power market, our linear model in (2) can be justified as shown in [19], [20]. Also for the upper level, there is no optimization. Instead, the market or aggregator just provides the pricing feedback to the lower level, so that the smart building consumers can change their strategies. III. D IFFERENTIAL G AME A NALYSIS If a centralized control of all buildings is feasible, then one can formulate a stochastic dynamic optimization problem to control the operation of the battery storage and air conditioner units in all buildings so as to maximize the aggregate utility of all users. For each user, the utility function depends on both the cost of electricity and the occupants’ comfort based on the home’s temperature. An alternative approach is to apply game theory to a distributed optimization framework for each smart building based on local information only. Such framework would be able to address some of key optimization challenges regarding energy efficiency including heterogeneous nature of building ECS systems, complexity of interactions among smart buildings and non-linear formulated optimization problems. Note that, centralized optimization across all buildings is not practical due to the larger computational burden compared to complexity of distributed optimization problem for each building. The centralized approach computation complexity may grow polynomially or exponentially with respect to the number of buildings controlled. Next we explain our proposed differential game formulation and discuss some of its properties. In particular, we prove that the optimal solutions constitute a feedback Nash equilibrium for the formulated game.
B. Upper Level (Market or Aggregator) Define the total bus load vector UD 0 =[UD1 , ..., UDi , ..., UDM ] , where M is the total number of feeders and UDi is the total number of load of buildings connected to feeder i. At the beginning of each time interval1 , given the demand vector UD from all M feeders that all buildings are connected to, a grid operator checks total available generation in the market and determines the price. Considering an estimated quadratic cost function for the oligopolistic electricity market [19] yields the electricity spot price as a linear function of aggregated building consumptions [20]. This model would help to study the impact of large-scale
A. Game Formulation At any time t, the stochastic differential game of each smart building i is to control the battery output used for building usage, ui1 (t), and the air conditioner electricity usage, ui2 (t),
1 Without
loss of generality, we assume the time intervals is one hour in this paper. Other time interval can be implemented in a similar way
3
so as to minimize the cost. We model the cost at time t as a quadratic function of the building power consumption: i
µ (ui1 (t), ui2 (t)) = p(t)[li (t) − ui1 (t) + ui2 (t)] + η i [xi2 (t) − xid ]2 = N X 1 [αpc (t) + (li (t) − ui1 (t) + ui2 (t) − g i (t))] α i=1 [li (t) − ui1 (t) + ui2 (t)] + η i [xi2 (t) − xid ]2 ,
where A and B are diagonal matrix with diagonal entries: A = diag[−β 1 , 1 , . . . , −β N , N ]1×2N ;
(4)
and B=
(12) 1
1
where the first term represents the cost of the building electricity consumption, and the second term models penalty of temperature differences from the desired temperature, xid . The variable η i scales the temperature’s penalty value. By minimizing the objective function in (4), we achieve the optimal policies that can balance the trade-off between user comfort and electricity cost minimization by controlling the HVAC usage and local energy storage, given the current states of the system which follows the dynamics in (1). Next, we introduce the expected utility function Li of each building over the random nature of renewable energy during a time period of interest, e.g., one day, as follows: (Z ) T i i i L = Ew µ (t)dt + h (x(T )) , (5)
l1 2 γ 1 K 1 (l1 ) ) 2
Li .
(1 − 1 )(t1OD + .. C= . N e +
+ 1 x1d
U= uN 1
−
−
ψ, −uN 2
−
ψ, . . . , ui1
l 2 γ N K N (lN ) ) 2
+ N x N d
,
(13)
2N ×1
.
(14)
µi (X(t), U(t)) = 1 [X(t)T Qis X(t) + U(t)T Ri (t)U(t)]. 2 Finally, for the value function in (6), we have: vi (X, U, t)
=
(15)
min Li
(16) )
U(t)
(Z =
min Ew U(t)
(6)
T
µi (X(t), U(t))dt + hi [X(T )] ,
0
where hi [X(T )] =
(7)
N T X = [x11 , x12 − xd , x21 , x22 − xd , . . . , xi1 , xi2 − xd , xN 1 , x2 − xd ] (8) where [·]T means transposition, and
ψ, −u12
N
The cost function in (4) can be rewritten as follows:
To convert our stochastic differential optimization problem into a linear quadratic format, we use changes of variables
[u11
N
1×2N
Without loss of generality, we assume that hi (x(T )) = v i (x(T ), u, T ) = 0.
N
h iT ρ = 1, 0, 1, . . . , 1, 0
where Ew {·} is used to denote the expected value with respect to the random variable w, T denotes the final point of time to be evaluated in the period of scheduling and h(x(T )) is the terminal condition. The value function of u11 (t) and u12 (t) can be written as min
1
e1 +
(1 − N )(tN OD +
0
ui1 (t),ui2 (t)
1
diag[−1, −γ K (1 − ), . . . , −1, −γ K (1 − )]1×2N ;
vi (x, u, t) =
(11)
li li − , −ui2 − , . . . , 2 2
− ψ],
where
0 . .. 0 1 1 Ri (t) = α 1 0 .. .
...
0
1 T i X Qsf X, 2
(17)
...
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
...
0
0
0
0
0
0
...
1
1
−1
1
1
...
...
1
−1
1
1
1 ...
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 ... .. . ...
0 .. . 0 1 1 0 .. .
...
0
0
0
0
0 ...
0
, (18)
2N ×2N
i
PN
i
i=1
ψ=
PN
where for building i, all rows in R are zero but 2i − 1 and 2i.
i
l − i=1 g + αpc (t) . 2(N − 1)
(9)
Qisf = 02N ×2N , and Qis is a diagonal matrix with
As a result, the game dynamics can be written in a matrix form as follows: ˙ = X =
Qis = diag[0, . . . , 0, η i , 0, . . . , 0].
f [X(t), U(t)] + ρw AX(t) + BU(t) + C + ρw,
(19)
(20)
The stochastic differential optimization problem in (16)-(20) has an affine quadratic format. Notice that (17) and (18) are
(10) 4
ζ i (T ) = 02N ×2N ,
zero in our case. We write in the format to consistent with the standard format [17]. Next, we use dynamic programming to derive the optimal control solution for each building.
mi (T ) = 0, and mi is obtained from: 0 1 −1 0 ˙ i (t) + γ i (t)ζ i (t) + ζ i (t)Bi Ri B i ζ i (t) = 0. m 2
B. Solution Based on Dynamic Programming Differential games are the extension of the basic optimal control problem and their analysis relies heavily on concepts and techniques in optimal control theory [17]. Equilibrium strategies in the feedback form are best studied by looking at a system of Hamilton-Jacobi-Bellman (HJB) equations for the value functions of various players. Using dynamic programming, the solution is obtained backwards in time. That is, we start at all possible final states with the corresponding final times. The optimal action at each final time is selected, we then proceed back one step in time and determine the optimal action at each stage. This process is repeated until the initial time or stage is reached. The core of dynamic programming when it is applied to continuous-time optimal control is the partial differential equation (PDE) of an HJB formulation. Now consider the stochastic differential game in (16)-(20), the optimal strategy can be derived using the stochastic HJB equation [17]: σ 2 ∂ 2 vi (X, U, t) ∂vi (X, U, t) = min + (21) − ∂t 2 ∂X 2 Ui (t) ∇X vi ((X, U, t))f i (X(t), U(t)) + µi (X(t), U(t)) .
γ i (t) = C0 − Bi Ri
∗
0
Zi (T ) = 02N ×2N , −1
Bi Zi (t).
−1
i
(30)
0
= −αR (t) B (t) ( ) 0 0 0 X(t) [(Zi (t)) + Zi (t)] + ζ i (t) . 2 For the proposed game in (10)-(16), Ri , i = 1, . . . , N , matrices in (18) are singular block matrices (therefore non– invertible). Using the Singular Value Decomposition (SVD) H factorization method, we have Ri = Ei Σi π i , where Ei i is a real unitary matrix, Σ is a rectangular diagonal matrix H with singular values of Ri on the diagonal, and π i (the conjugate transpose of π i ) is a real unitary matrix. Using SVD factorization, a Moore–pseudoinverse of matrix Ri can + + H be calculated as Ri = π i Σi Ei . Using Moore–Penrose pseudoinverse of matrix Ri in (30), the following Lemma shows that player i does not need knowledge of other players’ states. Lemma 1. For each player i, the associated columns of the other players in Moore-Penrose pseudoinverse matrix of Ri are zeros. The proof is shown in Appendix. From this Lemma, re+ placing Ri in (30), the elements of the state vector related to other player are canceled out by multiplication to zero columns. Therefore, each player requires only its own state information to calculate the optimal action and the overhead of feedback information reduces significantly. After calculation of optimal actions from (30), to obtain the original optimal control decisions u∗ , the following change of variables should be used: u∗ =
(31) PN
[U11 + (23)
−(U12 +
(24)
where Fi (t) = A − Bi Ri
(29)
T
0
i
0
B i Zi (t) + Qis = 0,
T
Bi ζ i (t),
Ui (t, X) = −αRi (t)−1 Bi (t) ∇X vi (t, X)
Z˙ i (t) + Zi (t)Fi (t) + Fi (t)Zi (t) −1
−1
(28)
Finally, the optimal control strategy can be obtained as
In general, the HJB equation does not have a classical (smooth) solution. Although some efforts have been made in the past, e.g., to obtain the viscosity solution [21], or the minimax solution [22]. However, for the special case of a affine-linear quadratic game, where the system dynamics are described by a set of linear differential equations and the cost function is quadratic, the value function has the unique solution which should satisfy a set of first order differential equations. Therefore, a closed form solution for the optimal action can be obtained for this special case. According to [17], the value function for an affine-linear quadratic problem has the following solution for v(t): 0 1 vi (X, U, t) = X(t) Zi (t)X(t)+X(t)T ζ i (t)+ξ i (t)+mi (t), 2 (22) 0 where X(t) is the complex conjugate of X(t), and Zi satisfies the following Riccati differential equations:
+Zi (t)Bi Ri
(27)
(25)
ζ i and mi can be obtained from the following differential equations, respectively: −1 0 ζ˙ i (t) + Fi (t)ζ i (t) + Zi (t)Bi Ri B i ζ i (t) + Zi (t)Bi = 0, (26)
5
i
l −
PN
i
i=1 g + αpc (t) , 2(N − 1) PN i PN i j6=i l − i=1 g + αpc (t) j6=i
) 2(N − 1) li li , . . . , U1i + , −(U2i + ) 2 2 PN i PN i l − j6=i i=1 g + αpc (t) , . . . , U1N + , 2(N − 1) PN i PN i j6=i l − i=1 g + αpc (t) N −(U2 + )]. 2(N − 1)
TABLE I B UILDING C ONSUMPTION S CHEDULING A LGORITHM
TABLE II N UMBER OF B UILDINGS C ONNECTED TO E ACH B US
For each hour t=1:24 Update the market price according to (2). Compute ζ using (26). Compute vector of optimal decisions, U ∗ , according to (30). Use change of variables in (31) to transform U ∗ to u∗ . Update the total hourly demand. Send back the total hourly demand to the market. End
bus 1 0 bus 8 0
bus 2 2100 bus 9 2950
bus 3 9400 bus 10 900
bus 4 4800 bus 11 350
bus 5 760 bus 12 6100
bus 6 1120 bus 13 1350
bus 7 0 bus 14 1490
Next, each building reports its total consumptions to the upper level, and the market makes its decisions based on the total bus load vector UD . In summary, the daily building load control algorithm is shown in Table I. C. Properties and Discussion In this section we show that the optimal control solution constitutes a feedback Nash equilibrium to the stochastic differential game. For the proposed game, the N -tuple strategies that are defined below constitute a feedback Nash equilibrium solution [17]. Definition 1. For an N-person game as defined in (8)-(20), a set of controls U ∗ (t, X), ∀i = 1, . . . , N , constitutes a feedback Nash equilibrium of the formulated differential game if there exists functions vi (X, t), ∀i = 1, . . . , N , that satisfy the following relations: ∗
∗
vi (X , U , t) ≥ vi (X, U, t)
Fig. 2. The fourteen bus system studied in simulations.
Next, to prove that our proposed game has the feedback Nash equilibrium, we note that the differential game in (10)(16) is in the affine linear form. Therefore, the value function in (22) is twice continuously differentiable and the derived optimal solutions by the HJB equations can indeed characterize the feedback Nash equilibrium.
(32)
where ∗
∗
∗
∗
U = [U11 , U21 , · · · , U1i , U2i , · · · , U1N , U2N ], ∗
∗
∗
∗
∗
∗
U∗ = [U11 , U21 , · · · , U1i , U2i , · · · , U1N , U2N ].
(33)
IV. S IMULATION R ESULTS
(34)
In this section, we numerically investigate the performance of the proposed stochastic differential game to confirm and complement the results presented in the previous sections. Consider the IEEE 14-bus test system in Fig. 2, where the load at each bus is the summation of several homogeneous smart building load at that bus and we consider the DC case. The number of buildings that are connected to each bus is shown in Table II. The bus and the line parameters are set according to the model in [23]. According to [24], the leadacid batteries which are suitable for energy-smart buildings are generally 85-95 % efficient. Therefore, for simulation purposes the value for the leakage rate of the batteries is considered to be β = 0.05. The value for as the factor of inertia of the building is a function of the time constant of the building which can be defined as the energy stored per unit area in the construction per unit change in heat flux. Finally, the overall thermal conductivity is calculated based on the real data for a typical building in Texas provided by [25] as 0.5 watts per meter kelvin. In our simulation model, an upper band of 0.6kW for power consumption of HVAC units is considered.
Based on the definition above, we want to show our proposed game converge to the feedback Nash equilibrium. If there exists a function v i (X, t) that is twice continuously differentiable, then the two partial differential feedback Nash equilibrium solutions in continuous time can be characterized using the stochastic HJB equations, which are necessary conditions of the candidate optimal control strategy. The proof is well-described in [17], and therefore is not included in this manuscript Theorem 1. A set of feedback strategies U ∗ (t, X) leads to a feedback Nash equilibrium for stochastic differential game in (8)-(20), and X ∗ (t), 0 ≤ t ≤ T is the corresponding state trajectory, if there exist suitably smooth functions v i (t) , for i = 1, . . . , N satisfying the following rectilinear parabolic partial differential equations: n 2 2 i i i σ ∂ v (X,U,t) = min + ∂v (X,U,t) − ∂v (X,U,t) ∂t 2 ∂X2 ∂X i U (t) f (X(t), U(t)) + µi (X(t), U∗ (t)) . (35) 6
92
0.063
90 Outdoor Temerature (Fahrenheit)
0.062
Price ($/kWh)
0.061
0.06
0.059
88 86 84 82 80 78
0.058
76
0.057
0.056
2
4
6
8
10
Day−Ahead Pricing Real Time Pricing 2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
12 14 Time (Hour)
16
18
20
22
24
(a) Outdoor Temperature
24
5 4.5 Wind Turbine Output (kW)
Fig. 3. Daily price for two pricing scenarios.
The temperature’s penalty value η i = 1. The price factors in (2) are set as ∀t, pc (t) = 0.055 kWh and α = 1. To study how the proposed demand response method affects electricity scheduling at the buildings level, we compare the performance of our algorithm with a more realistic day-ahead pricing scenario. We assume that the day-ahead electricity prices are fully known to the ECS. Fig. 3 illustrates the day-ahead and real-time electricity price from data in [26]. We also show the real-time pricing in the same Fig. For simplicity, we focus on one building at bus 2 as an example. The outdoor temperature, the mean wind turbine output, and the uncontrollable load are depicted in Fig. 4(a), Fig. 4(b) and Fig. 4(c), respectively [26]. In Fig. 5(a) and Fig. 5(b), the daily states of the considered building for the mentioned three methods are depicted. For all three scenarios, as price tends to increase, the battery tends to discharge in order to cover a portion of the building power consumption. Furthermore, the indoor temperature tends to increase due to lowering the air conditioner’s load during peak hours. We can also see that, for all scenarios, the variations of both battery level and the indoor temperature are correlated to the changes in price values. Here, the average usage from the battery reduces by around 10% for real-time pricing in peak hours (hours 18-20) compared to the day-ahead pricing case. Next, we compare our proposed joint real-time pricing at upper level and demand response at lower level, with the two other design scenarios. The corresponding daily load profiles are shown in Fig. 6. For both day-ahead and real-time pricing techniques, the peak load is reduced at around 8:00 PM. However, the use of the proposed real-time pricing technique yields a more flat load shape compared to other methods. Fig. 7 studies how the PAR in aggregate load varies as the mean of the daily outdoor temperature increases for three pricing scenarios. As it is shown in Fig. 6 compared to the day-ahead pricing technique, the proposed real pricing technique makes the load curve more flat. Therefore, the PAR of associated load curve is also have a smaller value.
4 3.5 3 2.5 2 1.5 1
5
10
15
20
Time (Hour)
(b) Predicted Output of Wind Turbine
Building Uncontrollable Load (kW)
2.75
2.7
2.65
2.6
2.55
5
10
15
20
Time (Hour)
(c) Uncontrollable Load Fig. 4. Region characteristics at Bus 2.
The mean daily cost of the considered building versus the mean daily outdoor temperature is shown in Fig. 9. We can see that the real-time pricing combined with the proposed stochastic differential game can reduce the daily cost. The price value under the real-time pricing has higher gradient since all building users increase their power consumption in the case of a high temperature, which results in a higher price. For temperatures higher than 100◦ F the optimal choice of power consumed by each HVAC unit reaches the upper band (0.6 kW) for both day-ahead and real-time pricing to maintain the indoor temperature in the desired range. That is why the result does not change beyond 100◦ F . Finally the impact of buildings on price is studied in Fig. 8 by comparing the average daily consumption of a sample 7
8
1.06 Day−Ahead Pricing Real Time Pricing
Day−ahead Pricing Real Time Pricing
7.8
1.05
7.4 7.2
1.04
7
Average PAR
Battery Storage Level (kW)
7.6
6.8 6.6 6.4
1.03
1.02
6.2 6
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
1.01
24
(a) Battery Storage Level
1
80
85 90 Outdoor Temperature (Fahrenheit)
95
100
71.5 Day−ahead Pricing
71
Fig. 7. Comparison of load PAR for two pricing scenarios.
Real Time Pricing
Indoor Temperature (Fahrenheit)
70.5
70
Average Daily Power Consumption of a Building (kW)
35 69.5
69
68.5
68
67.5
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
24
(b) Indoor Temperature Fig. 5. System states for a sample building at Bus 2.
30
25
20
15
10
5
0 34.5
1
2
3 α
4
5
34
Fig. 8. Impact of buildings power consumption factor α in (2) on price
Load (MW)
33.5
V. C ONCLUSION
33
In this paper, we developed a stochastic differential game model for autonomous demand response when the price of electricity varies during the day. The model explains how endusers can decrease their electricity bill when having dynamic load, where the load dynamics formulated as differential equations. Real-time pricing also gives the aggregator the opportunity to influence end users’ load profile through pricing of power. We studied the interaction between the market and buildings using a two-level differential game model. To gain insights, two dynamic states are particularly investigated: the battery’s state-of-charge and the room temperature. The HJB equation is used to study the solution of the formulated game among different buildings. As simulation results over three different pricing scenarios show, the proposed method reduces the overall power consumption of all users, by storing the energy when the price is low and by later discharging it when the price is high. The peak-to-average ratio in aggregate load demand as well as overall energy cost are also greatly reduced.
32.5
32 Real Time Pricing Day−ahead Pricing 31.5
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
24
Fig. 6. Total daily load at Bus 2.
building at Bus no. 2 for different values of α in (2). Increasing α reduces the impact of buildings on spot price and yields an almost constant price. However, since HVAC units are considered as the only controllable load in system model, the consumption would not increase beyond certain range for α ≥ 4. 8
zero N × N matrix; 2) non zeros columns are in matrix Ri2 and matrix Ri1 is a zero N × N matrix; 3) both Ri1 and Ri2 matrices have just one non-zero rows. For all these 3 cases, the necessary conditions in (37-39) are satisfied. Therefore + + + Ri = (Ri1 Ri2 ). For all 3 cases, zero rows in Ri1 and Ri2 + + are associated with zero columns in Ri1 and Ri2 . Therefore, + all columns in Ri except column i and i + 1 are zeros and cancel out the state information of the other players but player i.
320
300
Electricity Bill ($)
280
260
240
220
R EFERENCES Day−Ahead Pricing
200
Real Time Pricing 180
80
85 90 Outdoor Temperature (Farenheit)
95
100
Fig. 9. Daily cost vs. outdoor temperature for two pricing scenarios.
ACKNOWLEDGEMENT This paper is partially supported by the US NSF CNS0910461, CNS-0905556, CNS-0953377, ECCS-1028782, CNS-1117560, CNS-1265268, ECCS 1253516, Qatar national research fund, and Electric Power Analytics Consortium at the University of Houston. We would also want to thank Dr. Rong Zheng in the computing and software department of the McMaster University, Canada for initial discussion. A PPENDIX -P ROOF OF L EMMA 1: Proof. For each player i, matrix Ri is written as follows: 0 . .. 0 1 i R (t) = 1 0 .. . 0
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
...
0
0
0
0
0
0
...
1
1
−1
1
1
...
...
1
−1
1
1
1 ...
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 ... .. . ...
0 .. . 0 1 1 0 .. .
...
0
0
0
0
0 ...
0
, (36)
2N ×2N
where all rows are zeros except row i and i + 1. According to Theorem 3 in [27], if matrix A is partitioned row-wise as + A = (A1 :: A2 )T , the matrix M of form M = (A+ 1 A2 ) = + A if the following relationships are satisfied: A+ 1 A2 = 0,
A+ 2 A1 = 0,
(37)
+ ∗ ∗ (A1 A+ 1 ) (A2 A2 ) = 0,
(38)
+ A1 A+ 1 A2 A2 = 0.
(39)
i
Matrix R can have 3 case of row-wise decomposition : 1) non zeros columns are in matrix Ri1 and matrix Ri2 is a 9
[1] N. Ruiz, I. Cobelo, and J. Oyarzabal, “A direct load control model for virtual power plant management,” IEEE Trans. Power Syst., vol. 24, no. 2, pp. 959–966, May 2009. [2] A. Gomes, C. H. Antunes, and A. G. Martins, “A multiple objective approach to direct load control using an interactive evolutionary algorithm,” IEEE Trans. Power Syst., vol. 22, no. 3, pp. 1004-1011, August 2007. [3] C. M. Chu, T. L. Jong, and Y. W. Huang, “A direct load control of air-conditioning loads with thermal comfort control,” in proceedings of IEEE PES Gen. Meet., San Francisco, CA, June 2005. [4] E. Hossain, Z. Han, and V. Poor, Smart Grid Communications and Networking, Cambridge University Press, UK, 2012. [5] A. Mohsenian-Rad and A. Leon-Garcia. “Optimal residential load control with price prediction in real-time electricity pricing environments,” IEEE Transactions on Smart Grid, vol. 1, no. 2, pp. 120-133, September 2010. [6] D. Fudenberg and J. Tirole,Game Theory. MIT Press, 1993. [7] A. C. Tellidou and A. G. Bakirtzis, “Demand response in electricity markets,” in Proc. 15th Int. Conf. Intell. Syst. Appl. Power Syst., pp. 1-6, Curitiba, November 2009. [8] Y. Chen, W. S. Lin, F. Han; Y. H. Yang, Z. Safar, K. J. R. Liu, “A cheatproof game theoretic demand response scheme for smart grids,” 2012 IEEE International Conference on Communications (ICC), pp. 33623366, Ottawa, June 2012. [9] C. Chen, S. Kishore, W. Zhifang, M. Alizadeh, A. Scaglione, “How will demand response aggregators affect electricity markets? - A Cournot game analysis,” 5th International Symposium on Communications Control and Signal Processing (ISCCSP), pp. 1-6, May 2012. [10] S. Dave, M. Sooriyabandara, and L.Y. Zhang, “Application of a gametheoretic energy management algorithm in a hybrid predictive-adaptive scenario,” 2rd IEEE International Conference on Innovative Smart Grid Technologies, Manchester, December 2011. [11] J. Chen, B. Yang, and X. Guan, “Optimal demand response scheduling with Stackelberg game approach under load uncertainty for smart grid,” IEEE Third International Conference on Smart Grid Communications (SmartGridComm), Tainan, Taiwan, pp. 546-551, November 2012. [12] Z. Chen, L. Wu, and Y. Fu, “Real-Time Price-Based Demand Response Management for Residential Appliances via Stochastic Optimization and Robust Optimization,” IEEE Transactions on Smart Grid, vol. 3, issue 4, pp. 1822-1831, December 2012. [13] B. Jiang, and F. Yunsi, “Dynamic Residential Demand Response and Distributed Generation Management in Smart Microgrid with Hierarchical Agents,” Energy Procedia, vol. 12, pp. 76–90, December 2011. [14] A. H. Mohsenian-Rad, V. Wong, J. Jatskevich, R. Schober, and A. LeonGarcia, “Autonomous demand-side management based on gametheoretic energy consumption scheduling for the future smart grid,,” IEEE Trans. Smart Grid, vol. 1, no. 3, pp. 320-331, December 2010. [15] H. K. Nguyen, J. B. Song, and Z. Han, “Demand side management to reduce peak-to- average ratio using game theory in smart grid,” IEEE INFOCOM Workshop on Green Networking and Smart Grids, March 2012. [16] W. Saad, Z. Han, H. V. Poor, and T. Basar, “game-theoretic methods for the smart grid: an overview of microgrid systems, demand-side management, and smart grid communications,” IEEE Signal Processing Magazine, special issue on Signal Processing Techniques for Smart Grid, vol. 29, no. 5, p.p. 86-105, Yongin, South Korea, September 2012. [17] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed. London, U.K.: Academic, 1995.
[18] P. Constantopoulos, F. C. Schweppe, R. C. Larson, “ESTIA: A RealTime Consumer Control Scheme for Space Conditioning Usage under Spot Electricity Pricing,” Computers and Operations Research , vol. 18, no. 8, pp. 751-765, 1991. [19] C. E. Lin and G. L. Viviani,“ Hierarchical economic dispatch for piecewise quadratic cost functions,” IEEE Trans. Power App. Syst., vol.103, no. 6, pp. 11701175, June 1984. [20] N. Li, L. Chen, and S. H. Low, “Optimal demand response based on utility maximization in power networks,” in IEEE Power and Energy Society General Meeting, July 2011. [21] M. G. Crandall, and P. L. Lions, “Viscosity Solutions of Hamilton-Jacobi equations” Transactions of the American Mathematical Society, vol. 277, no. 1, pp. 1-42, May 1983. [22] A. I. Subbotin, “Minimax solutions to the Hamilton Jacobi Equations” Journal of Mathematical Sciences, vol. 103, no. 6, pp. 772-777, March 2001. [23] R. D. Zimmerman, C. E. Murillo-Snchez, and R. J. Thomas, “SteadyState Operations, Planning and Analysis Tools for Power Systems Research and Education,” IEEE Transactions on power systems , vol. 26, no. 1, pp. 12–19, February 2011. [24] A. Mishra, D. Irwin, P. Shenoy, J. Kurose, and T. Zhu. “SmartCharge: Cutting the Electricity Bill in Smart Homes with Energy Storage,” In eEnergy, Proceedings of the 3rd International Conference on Future Energy Systems: Where Energy, Computing and Communication Meet, pp. 21-29, Madrid, May 2012. [25] P. Constantopoulos, F. C. Schweppe, R. C. Larson, “ESTIA: A RealTime Consumer Control Scheme for Space Conditioning Usage under Spot Electricity Pricing,” Computers and Operations Research, vol. 18, no.8, pp. 751-765, June 1991. [26] http://www.amigoenergy.com. [27] J.K. Baksalary, O.M. Baksalary, “Particular formulae for the MoorePenrose inverse of a columnwise partitioned matrix,” Linear Algebra and Its Applications no. 421, pp.16-23, 2007.
10
Electrical and Computer Engineering Department, University of Houston, Houston, TX, USA + Department of Electrical Engineering, University of California, Riverside, CA, USA
I. I NTRODUCTION
Abstract—Demand Response (DR) programs are implemented to encourage consumers to reduce their electricity demand when needed, e.g., at peak-load hours, by adjusting their controllable load. In this paper, our focus is on controllable load types that are associated with dynamic systems and can be modeled using differential equations. Examples of such load types include heating, ventilation, and air conditioning (HVAC), water heating, and refrigeration. In this regard, we propose a new demand response model based on a two-level differential game framework. At the beginning of each demand response interval, the price is decided by the upper level (aggregator, utility, or market) given the total demand of users in the lower level. At the lower level, for each player (residential or commercial buildings that are equipped with automated load control systems and local renewable generators), given the price from the upper level, the electricity usage of air conditioning unit and the battery storage charging/discharging schedules are controlled in order to minimize the user’s total electricity cost. The optimal user strategies are derived using the stochastic Hamilton-JacobiBellman equations. We also show that the proposed game can converge to a feedback Nash equilibrium. Based on the effect of real-time pricing on users’ daily demand profile, the simulation results demonstrate the properties of the proposed game and show how we can optimize consumers’ electricity cost in presence of time-varying prices.
Demand Response (DR) programs are implemented by utility companies to control the energy consumption at the customer side of the meter. Two popular DR approaches are direct load control (DLC) and smart pricing. In DLC [1]–[4], an aggregator can remotely control the operations and energy consumption of certain consumer appliances. In contrast, in smart pricing, users are encouraged to individually and voluntarily manage their load, e.g., by reducing their consumption at peak price hours. This can be done using automated Energy Consumption Scheduling (ECS) devices [5]. For each user, the ECS finds the best load schedule to minimize the user’s electricity cost while fulfilling the user’s energy needs. This can lead to autonomous demand response programs that burden a minimal control overhead on utilities. A common analytical tool to study autonomous DR systems is game theory [6], that provides a framework to study rational interactions and outcome in a distributed manner. In [7], a stochastic game is developed to model an hourly energy auction in which generators and consumers participate as adaptive agents. In [8], authors proposed a game theoretic demand response scheme to develop a distributed load prediction system that involves user participation. Authors in [9] employed the Cournot game model to analyze the market effect of a demand response aggregator on both shifting and reducing deferrable loads. Authors in [10] developed a hybrid day-ahead and real-time consumption scheduling for a number of houses that participate in a demand side program based on game theory. The interaction between the service provider and the users is modeled as a Stackelberg game in [11] to derive the optimal real-time electricity price and each user’s optimal power consumption. In [12], the real-time pricebased demand response (DR) management is evaluated for residential appliances via stochastic optimization and robust optimization approaches. The work in [12] considers considers uncertainties in real-time electricity prices and determines the optimal operation of residential appliances within 5-minute time. In [13], a dynamic Demand Response (DR) and Distributed Generation (DG) management approach is proposed in the context of smart microgrid for a residential community. The DG management coordinates with DR and considers stochastic elements, such as stochastic load and wind power, to reduce overall energy consumption cost. In [14], a residential energy consumption scheduling framework is proposed, which
Keywords: Stochastic differential game, autonomous demand response, smart building, real-time pricing.
u1 u2 x1 x2 xdi w e l β γ tOD p d g µ L h v N
N OMENCLATURE Power draw from battery for building usage. Air conditioner usage of electricity. The energy stored in the battery array. The indoor temperature of building. The desired temperature. The renewable output prediction. The prediction error. The uncontrollable load of a building. Battery leakage rate. The factor of inertia. The coefficient of performance. The outside temperature. Market spot price. Total power consumption of the building. Total power generation of the building. Building utility function. Building expected total utility function. Terminal condition. Building value function. Number of buildings. 1
attempts to achieve a desired trade-off between minimizing the electricity payment and minimizing the waiting time for the operation of household appliance in presence of real-time prices using price prediction. In [15], game theory is used for demand-side management to reduce the peak-to-average ratio in aggregate load demand. In [16], a tutorial is given for the game-theoretic methods on microgrid systems, demand-side management, and smart grid communications. Different from the prior work in [7]- [16], in this paper, we focus on game-theoretic analysis of price-based DR programs where controllable load types are associated with dynamic systems and can be modeled using differential equations. Examples of such loads include heating, ventilation, and air conditioning (HVAC), water heating, and refrigeration. In particular, we apply techniques from stochastic differential games [17]. To the best of our knowledge, this paper is the first work to study dynamic loads, such as HVAC systems, with differential equations in the context of a game-theoretic demand side management framework. The major efforts and results in this paper can be summarized as follows: 1) We study the strategic interactions between a Nash Cournot electricity market and multiple energy-smart buildings to construct a two-level stochastic differential game framework. At the upper level, the market offers a vector of hourly prices to end users. At the lower level, the energy-smart buildings as the lower level participate in demand response by managing controllable dynamic load in response to hourly prices set by the market. 2) We focus on smart buildings equipped with renewable resources generators, local energy storage and controllable HVAC units, in which users are able to respond to real-time grid conditions like electricity prices and weather conditions in order to minimize their cost. 3) We derive the optimal closed-form control strategies for each energy-smart building obtained by solving stochastic the Hamilton-Jacobi-Bellman (HJB) equation. We analyze the outcome of interactions between two levels and constitute a feedback Nash equilibrium solution. 4) Comparing to day-ahead pricing, the proposed technique makes the load profile more flat and reduces the peakto-average ratio (PAR) of aggregate load. 5) Using simulation results we show that by implementing our proposed stochastic differential DR game model, we can minimize the electricity cost of buildings. The rest of this paper is organized as follows. The system model is described in Section II. The stochastic differential game is constructed and it solution to the proposed game is derived in Section III. Simulation results are presented in Section IV. Conclusions are drawn in Section V.
Fig. 1. The interactions between the aggergator and individual buildings.
the market decides on a price to pass on to the end-users in the lower level, based on the total demand data from the lower level during the last time interval. The ECS unit of each building minimizes the cost of electricity consumption. Since there are multiple buildings competing for the electricity resources, the system can be analyzed using game theory [6]. A. Lower Level (Smart Building Consumers) Consider a total of N energy-smart buildings that participate in demand response program. Each building has two specific controllable loads: an air conditioner with a controllable thermostat, and an always-connected battery. We also assume that a renewable source of energy, e.g., a residential wind turbine or a roof-top solar panel, is available in each building, with its generated output to be used to charge the battery. Uncontrollable appliances with a total and known consumption of l(t) constitute the rest of the building power consumption. Given the price that is a function of the optimal strategy of the upper level player (see Section II-B), the decision variables available for consumers at each building i = 1, . . . , N are: ui1
=
power draw from battery for home usage,
ui2
=
air conditioner usage of electricity,
and the dynamic states include: xi1
= the energy stored in the battery array,
xi2
= the indoor temperature of the home.
The output power of the renewable generator is random and in our analysis, it is modeled as W + ei , where W denotes the renewable output prediction that is obtained using a day-ahead forecasting method and ei denotes the prediction error which is a Gaussian random variable with zero mean and variance σ 2 . As an example, the amount of power generated by a wind turbine can be modeled as a function of wind speed. As for the outside temperature, we assume that its day-ahead predictions are used based on standard weather forecasting data.
II. S YSTEM M ODEL In this section, we explain the system model that incorporates the impact of demand response on both supply and demand sides when real-time pricing is used. As illustrated in Fig. 1, we study a two-level design framework: at the upper level, at the beginning of each time interval, e.g., at each hour, 2
buildings’ power generation and consumption on spot price, p, as follows: PN PN [ j=1 dj (t) − j=1 g j (t)] p(t) = pc (t) + , (2) α where pc (t) is a constant price factor decided by market hourly, dj and g j are the electricity consumption and generation of building j respectively and α is a scalar parameter. Increasing α reduces the impact of buildings on spot price. From Section II-A, for each smart building i, the total power consumption can be calculated as
For each smart building, the dynamics of the states can be modeled using the following differential equations: " #" # " # x˙i1 −β i 0 xi1 = + (1) 0 (i − 1) xi2 x˙i2 " #" # −1 0 ui1 + 0 −γ i (1 − i )K i ui2 " # " # ei 1 w+ . (1 − i )tiOD 0 The differential equation in the first row in (1) models the dynamic of the battery’s state-of-charge. The differential equation in the second row models the variation in the building’s indoor temperature. Here, β i denotes the battery leakage rate. As the battery dynamic equation shows the output of renewable resource W +ei acts as the input to the battery, and the amount of power that is discharged for usage in the building acts as the output of battery. The thermal model in (1) is based on the a building thermal model in [18]. Here, i is the factor of inertia of the building which is a function of time constant of the building and overall thermal conductivity, γ i is the coefficient of performance of the air conditioning unit to cool the air inside the building, tiOD is the outside temperature and K i is a constant that is depends on the performance of the air conditioning unit and the total thermal mass. The air conditioning unit uses power ui2 to cool down the home’s indoor temperature. Note that, in this model, our focus is only on the cooling scenario. The results for the heating scenario are similar and can be obtained by changing the sign of −γ i (1 − i )K i from negative to positive. Notice that for the other deferrable appliances such as dish washers, there is no dynamic model like the air temperature and battery. In other word, if the appliances have done the work, the future will not be affected. This is the assumption in most of existing demand side management literature. For such appliances, it is easy to extend our model in (1) by setting the first term of right hand side with zero memory. Since the major contribution for this paper is to provide solution for the appliances with dynamic models, we do not consider the deferrable appliances without memory in the rest of this paper.
di (t) = li (t) − ui1 (t) + ui2 (t).
(3)
where li (t) is the power consumption beyond HVAC and battery. di (t) can be obtained by smart meter, and ui1 (t) & ui2 (t) are known. So li (t) can be easily calculated and no future value is needed. Another physical interpretation of p(t) can be viewed as the small pertubation around price pc (t). In other word, it is Tylor first order expansion for power market pricing. Since each building’s power consumption is relatively small compared with the whole power market, our linear model in (2) can be justified as shown in [19], [20]. Also for the upper level, there is no optimization. Instead, the market or aggregator just provides the pricing feedback to the lower level, so that the smart building consumers can change their strategies. III. D IFFERENTIAL G AME A NALYSIS If a centralized control of all buildings is feasible, then one can formulate a stochastic dynamic optimization problem to control the operation of the battery storage and air conditioner units in all buildings so as to maximize the aggregate utility of all users. For each user, the utility function depends on both the cost of electricity and the occupants’ comfort based on the home’s temperature. An alternative approach is to apply game theory to a distributed optimization framework for each smart building based on local information only. Such framework would be able to address some of key optimization challenges regarding energy efficiency including heterogeneous nature of building ECS systems, complexity of interactions among smart buildings and non-linear formulated optimization problems. Note that, centralized optimization across all buildings is not practical due to the larger computational burden compared to complexity of distributed optimization problem for each building. The centralized approach computation complexity may grow polynomially or exponentially with respect to the number of buildings controlled. Next we explain our proposed differential game formulation and discuss some of its properties. In particular, we prove that the optimal solutions constitute a feedback Nash equilibrium for the formulated game.
B. Upper Level (Market or Aggregator) Define the total bus load vector UD 0 =[UD1 , ..., UDi , ..., UDM ] , where M is the total number of feeders and UDi is the total number of load of buildings connected to feeder i. At the beginning of each time interval1 , given the demand vector UD from all M feeders that all buildings are connected to, a grid operator checks total available generation in the market and determines the price. Considering an estimated quadratic cost function for the oligopolistic electricity market [19] yields the electricity spot price as a linear function of aggregated building consumptions [20]. This model would help to study the impact of large-scale
A. Game Formulation At any time t, the stochastic differential game of each smart building i is to control the battery output used for building usage, ui1 (t), and the air conditioner electricity usage, ui2 (t),
1 Without
loss of generality, we assume the time intervals is one hour in this paper. Other time interval can be implemented in a similar way
3
so as to minimize the cost. We model the cost at time t as a quadratic function of the building power consumption: i
µ (ui1 (t), ui2 (t)) = p(t)[li (t) − ui1 (t) + ui2 (t)] + η i [xi2 (t) − xid ]2 = N X 1 [αpc (t) + (li (t) − ui1 (t) + ui2 (t) − g i (t))] α i=1 [li (t) − ui1 (t) + ui2 (t)] + η i [xi2 (t) − xid ]2 ,
where A and B are diagonal matrix with diagonal entries: A = diag[−β 1 , 1 , . . . , −β N , N ]1×2N ;
(4)
and B=
(12) 1
1
where the first term represents the cost of the building electricity consumption, and the second term models penalty of temperature differences from the desired temperature, xid . The variable η i scales the temperature’s penalty value. By minimizing the objective function in (4), we achieve the optimal policies that can balance the trade-off between user comfort and electricity cost minimization by controlling the HVAC usage and local energy storage, given the current states of the system which follows the dynamics in (1). Next, we introduce the expected utility function Li of each building over the random nature of renewable energy during a time period of interest, e.g., one day, as follows: (Z ) T i i i L = Ew µ (t)dt + h (x(T )) , (5)
l1 2 γ 1 K 1 (l1 ) ) 2
Li .
(1 − 1 )(t1OD + .. C= . N e +
+ 1 x1d
U= uN 1
−
−
ψ, −uN 2
−
ψ, . . . , ui1
l 2 γ N K N (lN ) ) 2
+ N x N d
,
(13)
2N ×1
.
(14)
µi (X(t), U(t)) = 1 [X(t)T Qis X(t) + U(t)T Ri (t)U(t)]. 2 Finally, for the value function in (6), we have: vi (X, U, t)
=
(15)
min Li
(16) )
U(t)
(Z =
min Ew U(t)
(6)
T
µi (X(t), U(t))dt + hi [X(T )] ,
0
where hi [X(T )] =
(7)
N T X = [x11 , x12 − xd , x21 , x22 − xd , . . . , xi1 , xi2 − xd , xN 1 , x2 − xd ] (8) where [·]T means transposition, and
ψ, −u12
N
The cost function in (4) can be rewritten as follows:
To convert our stochastic differential optimization problem into a linear quadratic format, we use changes of variables
[u11
N
1×2N
Without loss of generality, we assume that hi (x(T )) = v i (x(T ), u, T ) = 0.
N
h iT ρ = 1, 0, 1, . . . , 1, 0
where Ew {·} is used to denote the expected value with respect to the random variable w, T denotes the final point of time to be evaluated in the period of scheduling and h(x(T )) is the terminal condition. The value function of u11 (t) and u12 (t) can be written as min
1
e1 +
(1 − N )(tN OD +
0
ui1 (t),ui2 (t)
1
diag[−1, −γ K (1 − ), . . . , −1, −γ K (1 − )]1×2N ;
vi (x, u, t) =
(11)
li li − , −ui2 − , . . . , 2 2
− ψ],
where
0 . .. 0 1 1 Ri (t) = α 1 0 .. .
...
0
1 T i X Qsf X, 2
(17)
...
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
...
0
0
0
0
0
0
...
1
1
−1
1
1
...
...
1
−1
1
1
1 ...
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 ... .. . ...
0 .. . 0 1 1 0 .. .
...
0
0
0
0
0 ...
0
, (18)
2N ×2N
i
PN
i
i=1
ψ=
PN
where for building i, all rows in R are zero but 2i − 1 and 2i.
i
l − i=1 g + αpc (t) . 2(N − 1)
(9)
Qisf = 02N ×2N , and Qis is a diagonal matrix with
As a result, the game dynamics can be written in a matrix form as follows: ˙ = X =
Qis = diag[0, . . . , 0, η i , 0, . . . , 0].
f [X(t), U(t)] + ρw AX(t) + BU(t) + C + ρw,
(19)
(20)
The stochastic differential optimization problem in (16)-(20) has an affine quadratic format. Notice that (17) and (18) are
(10) 4
ζ i (T ) = 02N ×2N ,
zero in our case. We write in the format to consistent with the standard format [17]. Next, we use dynamic programming to derive the optimal control solution for each building.
mi (T ) = 0, and mi is obtained from: 0 1 −1 0 ˙ i (t) + γ i (t)ζ i (t) + ζ i (t)Bi Ri B i ζ i (t) = 0. m 2
B. Solution Based on Dynamic Programming Differential games are the extension of the basic optimal control problem and their analysis relies heavily on concepts and techniques in optimal control theory [17]. Equilibrium strategies in the feedback form are best studied by looking at a system of Hamilton-Jacobi-Bellman (HJB) equations for the value functions of various players. Using dynamic programming, the solution is obtained backwards in time. That is, we start at all possible final states with the corresponding final times. The optimal action at each final time is selected, we then proceed back one step in time and determine the optimal action at each stage. This process is repeated until the initial time or stage is reached. The core of dynamic programming when it is applied to continuous-time optimal control is the partial differential equation (PDE) of an HJB formulation. Now consider the stochastic differential game in (16)-(20), the optimal strategy can be derived using the stochastic HJB equation [17]: σ 2 ∂ 2 vi (X, U, t) ∂vi (X, U, t) = min + (21) − ∂t 2 ∂X 2 Ui (t) ∇X vi ((X, U, t))f i (X(t), U(t)) + µi (X(t), U(t)) .
γ i (t) = C0 − Bi Ri
∗
0
Zi (T ) = 02N ×2N , −1
Bi Zi (t).
−1
i
(30)
0
= −αR (t) B (t) ( ) 0 0 0 X(t) [(Zi (t)) + Zi (t)] + ζ i (t) . 2 For the proposed game in (10)-(16), Ri , i = 1, . . . , N , matrices in (18) are singular block matrices (therefore non– invertible). Using the Singular Value Decomposition (SVD) H factorization method, we have Ri = Ei Σi π i , where Ei i is a real unitary matrix, Σ is a rectangular diagonal matrix H with singular values of Ri on the diagonal, and π i (the conjugate transpose of π i ) is a real unitary matrix. Using SVD factorization, a Moore–pseudoinverse of matrix Ri can + + H be calculated as Ri = π i Σi Ei . Using Moore–Penrose pseudoinverse of matrix Ri in (30), the following Lemma shows that player i does not need knowledge of other players’ states. Lemma 1. For each player i, the associated columns of the other players in Moore-Penrose pseudoinverse matrix of Ri are zeros. The proof is shown in Appendix. From this Lemma, re+ placing Ri in (30), the elements of the state vector related to other player are canceled out by multiplication to zero columns. Therefore, each player requires only its own state information to calculate the optimal action and the overhead of feedback information reduces significantly. After calculation of optimal actions from (30), to obtain the original optimal control decisions u∗ , the following change of variables should be used: u∗ =
(31) PN
[U11 + (23)
−(U12 +
(24)
where Fi (t) = A − Bi Ri
(29)
T
0
i
0
B i Zi (t) + Qis = 0,
T
Bi ζ i (t),
Ui (t, X) = −αRi (t)−1 Bi (t) ∇X vi (t, X)
Z˙ i (t) + Zi (t)Fi (t) + Fi (t)Zi (t) −1
−1
(28)
Finally, the optimal control strategy can be obtained as
In general, the HJB equation does not have a classical (smooth) solution. Although some efforts have been made in the past, e.g., to obtain the viscosity solution [21], or the minimax solution [22]. However, for the special case of a affine-linear quadratic game, where the system dynamics are described by a set of linear differential equations and the cost function is quadratic, the value function has the unique solution which should satisfy a set of first order differential equations. Therefore, a closed form solution for the optimal action can be obtained for this special case. According to [17], the value function for an affine-linear quadratic problem has the following solution for v(t): 0 1 vi (X, U, t) = X(t) Zi (t)X(t)+X(t)T ζ i (t)+ξ i (t)+mi (t), 2 (22) 0 where X(t) is the complex conjugate of X(t), and Zi satisfies the following Riccati differential equations:
+Zi (t)Bi Ri
(27)
(25)
ζ i and mi can be obtained from the following differential equations, respectively: −1 0 ζ˙ i (t) + Fi (t)ζ i (t) + Zi (t)Bi Ri B i ζ i (t) + Zi (t)Bi = 0, (26)
5
i
l −
PN
i
i=1 g + αpc (t) , 2(N − 1) PN i PN i j6=i l − i=1 g + αpc (t) j6=i
) 2(N − 1) li li , . . . , U1i + , −(U2i + ) 2 2 PN i PN i l − j6=i i=1 g + αpc (t) , . . . , U1N + , 2(N − 1) PN i PN i j6=i l − i=1 g + αpc (t) N −(U2 + )]. 2(N − 1)
TABLE I B UILDING C ONSUMPTION S CHEDULING A LGORITHM
TABLE II N UMBER OF B UILDINGS C ONNECTED TO E ACH B US
For each hour t=1:24 Update the market price according to (2). Compute ζ using (26). Compute vector of optimal decisions, U ∗ , according to (30). Use change of variables in (31) to transform U ∗ to u∗ . Update the total hourly demand. Send back the total hourly demand to the market. End
bus 1 0 bus 8 0
bus 2 2100 bus 9 2950
bus 3 9400 bus 10 900
bus 4 4800 bus 11 350
bus 5 760 bus 12 6100
bus 6 1120 bus 13 1350
bus 7 0 bus 14 1490
Next, each building reports its total consumptions to the upper level, and the market makes its decisions based on the total bus load vector UD . In summary, the daily building load control algorithm is shown in Table I. C. Properties and Discussion In this section we show that the optimal control solution constitutes a feedback Nash equilibrium to the stochastic differential game. For the proposed game, the N -tuple strategies that are defined below constitute a feedback Nash equilibrium solution [17]. Definition 1. For an N-person game as defined in (8)-(20), a set of controls U ∗ (t, X), ∀i = 1, . . . , N , constitutes a feedback Nash equilibrium of the formulated differential game if there exists functions vi (X, t), ∀i = 1, . . . , N , that satisfy the following relations: ∗
∗
vi (X , U , t) ≥ vi (X, U, t)
Fig. 2. The fourteen bus system studied in simulations.
Next, to prove that our proposed game has the feedback Nash equilibrium, we note that the differential game in (10)(16) is in the affine linear form. Therefore, the value function in (22) is twice continuously differentiable and the derived optimal solutions by the HJB equations can indeed characterize the feedback Nash equilibrium.
(32)
where ∗
∗
∗
∗
U = [U11 , U21 , · · · , U1i , U2i , · · · , U1N , U2N ], ∗
∗
∗
∗
∗
∗
U∗ = [U11 , U21 , · · · , U1i , U2i , · · · , U1N , U2N ].
(33)
IV. S IMULATION R ESULTS
(34)
In this section, we numerically investigate the performance of the proposed stochastic differential game to confirm and complement the results presented in the previous sections. Consider the IEEE 14-bus test system in Fig. 2, where the load at each bus is the summation of several homogeneous smart building load at that bus and we consider the DC case. The number of buildings that are connected to each bus is shown in Table II. The bus and the line parameters are set according to the model in [23]. According to [24], the leadacid batteries which are suitable for energy-smart buildings are generally 85-95 % efficient. Therefore, for simulation purposes the value for the leakage rate of the batteries is considered to be β = 0.05. The value for as the factor of inertia of the building is a function of the time constant of the building which can be defined as the energy stored per unit area in the construction per unit change in heat flux. Finally, the overall thermal conductivity is calculated based on the real data for a typical building in Texas provided by [25] as 0.5 watts per meter kelvin. In our simulation model, an upper band of 0.6kW for power consumption of HVAC units is considered.
Based on the definition above, we want to show our proposed game converge to the feedback Nash equilibrium. If there exists a function v i (X, t) that is twice continuously differentiable, then the two partial differential feedback Nash equilibrium solutions in continuous time can be characterized using the stochastic HJB equations, which are necessary conditions of the candidate optimal control strategy. The proof is well-described in [17], and therefore is not included in this manuscript Theorem 1. A set of feedback strategies U ∗ (t, X) leads to a feedback Nash equilibrium for stochastic differential game in (8)-(20), and X ∗ (t), 0 ≤ t ≤ T is the corresponding state trajectory, if there exist suitably smooth functions v i (t) , for i = 1, . . . , N satisfying the following rectilinear parabolic partial differential equations: n 2 2 i i i σ ∂ v (X,U,t) = min + ∂v (X,U,t) − ∂v (X,U,t) ∂t 2 ∂X2 ∂X i U (t) f (X(t), U(t)) + µi (X(t), U∗ (t)) . (35) 6
92
0.063
90 Outdoor Temerature (Fahrenheit)
0.062
Price ($/kWh)
0.061
0.06
0.059
88 86 84 82 80 78
0.058
76
0.057
0.056
2
4
6
8
10
Day−Ahead Pricing Real Time Pricing 2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
12 14 Time (Hour)
16
18
20
22
24
(a) Outdoor Temperature
24
5 4.5 Wind Turbine Output (kW)
Fig. 3. Daily price for two pricing scenarios.
The temperature’s penalty value η i = 1. The price factors in (2) are set as ∀t, pc (t) = 0.055 kWh and α = 1. To study how the proposed demand response method affects electricity scheduling at the buildings level, we compare the performance of our algorithm with a more realistic day-ahead pricing scenario. We assume that the day-ahead electricity prices are fully known to the ECS. Fig. 3 illustrates the day-ahead and real-time electricity price from data in [26]. We also show the real-time pricing in the same Fig. For simplicity, we focus on one building at bus 2 as an example. The outdoor temperature, the mean wind turbine output, and the uncontrollable load are depicted in Fig. 4(a), Fig. 4(b) and Fig. 4(c), respectively [26]. In Fig. 5(a) and Fig. 5(b), the daily states of the considered building for the mentioned three methods are depicted. For all three scenarios, as price tends to increase, the battery tends to discharge in order to cover a portion of the building power consumption. Furthermore, the indoor temperature tends to increase due to lowering the air conditioner’s load during peak hours. We can also see that, for all scenarios, the variations of both battery level and the indoor temperature are correlated to the changes in price values. Here, the average usage from the battery reduces by around 10% for real-time pricing in peak hours (hours 18-20) compared to the day-ahead pricing case. Next, we compare our proposed joint real-time pricing at upper level and demand response at lower level, with the two other design scenarios. The corresponding daily load profiles are shown in Fig. 6. For both day-ahead and real-time pricing techniques, the peak load is reduced at around 8:00 PM. However, the use of the proposed real-time pricing technique yields a more flat load shape compared to other methods. Fig. 7 studies how the PAR in aggregate load varies as the mean of the daily outdoor temperature increases for three pricing scenarios. As it is shown in Fig. 6 compared to the day-ahead pricing technique, the proposed real pricing technique makes the load curve more flat. Therefore, the PAR of associated load curve is also have a smaller value.
4 3.5 3 2.5 2 1.5 1
5
10
15
20
Time (Hour)
(b) Predicted Output of Wind Turbine
Building Uncontrollable Load (kW)
2.75
2.7
2.65
2.6
2.55
5
10
15
20
Time (Hour)
(c) Uncontrollable Load Fig. 4. Region characteristics at Bus 2.
The mean daily cost of the considered building versus the mean daily outdoor temperature is shown in Fig. 9. We can see that the real-time pricing combined with the proposed stochastic differential game can reduce the daily cost. The price value under the real-time pricing has higher gradient since all building users increase their power consumption in the case of a high temperature, which results in a higher price. For temperatures higher than 100◦ F the optimal choice of power consumed by each HVAC unit reaches the upper band (0.6 kW) for both day-ahead and real-time pricing to maintain the indoor temperature in the desired range. That is why the result does not change beyond 100◦ F . Finally the impact of buildings on price is studied in Fig. 8 by comparing the average daily consumption of a sample 7
8
1.06 Day−Ahead Pricing Real Time Pricing
Day−ahead Pricing Real Time Pricing
7.8
1.05
7.4 7.2
1.04
7
Average PAR
Battery Storage Level (kW)
7.6
6.8 6.6 6.4
1.03
1.02
6.2 6
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
1.01
24
(a) Battery Storage Level
1
80
85 90 Outdoor Temperature (Fahrenheit)
95
100
71.5 Day−ahead Pricing
71
Fig. 7. Comparison of load PAR for two pricing scenarios.
Real Time Pricing
Indoor Temperature (Fahrenheit)
70.5
70
Average Daily Power Consumption of a Building (kW)
35 69.5
69
68.5
68
67.5
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
24
(b) Indoor Temperature Fig. 5. System states for a sample building at Bus 2.
30
25
20
15
10
5
0 34.5
1
2
3 α
4
5
34
Fig. 8. Impact of buildings power consumption factor α in (2) on price
Load (MW)
33.5
V. C ONCLUSION
33
In this paper, we developed a stochastic differential game model for autonomous demand response when the price of electricity varies during the day. The model explains how endusers can decrease their electricity bill when having dynamic load, where the load dynamics formulated as differential equations. Real-time pricing also gives the aggregator the opportunity to influence end users’ load profile through pricing of power. We studied the interaction between the market and buildings using a two-level differential game model. To gain insights, two dynamic states are particularly investigated: the battery’s state-of-charge and the room temperature. The HJB equation is used to study the solution of the formulated game among different buildings. As simulation results over three different pricing scenarios show, the proposed method reduces the overall power consumption of all users, by storing the energy when the price is low and by later discharging it when the price is high. The peak-to-average ratio in aggregate load demand as well as overall energy cost are also greatly reduced.
32.5
32 Real Time Pricing Day−ahead Pricing 31.5
2
4
6
8
10
12 14 Time (Hour)
16
18
20
22
24
Fig. 6. Total daily load at Bus 2.
building at Bus no. 2 for different values of α in (2). Increasing α reduces the impact of buildings on spot price and yields an almost constant price. However, since HVAC units are considered as the only controllable load in system model, the consumption would not increase beyond certain range for α ≥ 4. 8
zero N × N matrix; 2) non zeros columns are in matrix Ri2 and matrix Ri1 is a zero N × N matrix; 3) both Ri1 and Ri2 matrices have just one non-zero rows. For all these 3 cases, the necessary conditions in (37-39) are satisfied. Therefore + + + Ri = (Ri1 Ri2 ). For all 3 cases, zero rows in Ri1 and Ri2 + + are associated with zero columns in Ri1 and Ri2 . Therefore, + all columns in Ri except column i and i + 1 are zeros and cancel out the state information of the other players but player i.
320
300
Electricity Bill ($)
280
260
240
220
R EFERENCES Day−Ahead Pricing
200
Real Time Pricing 180
80
85 90 Outdoor Temperature (Farenheit)
95
100
Fig. 9. Daily cost vs. outdoor temperature for two pricing scenarios.
ACKNOWLEDGEMENT This paper is partially supported by the US NSF CNS0910461, CNS-0905556, CNS-0953377, ECCS-1028782, CNS-1117560, CNS-1265268, ECCS 1253516, Qatar national research fund, and Electric Power Analytics Consortium at the University of Houston. We would also want to thank Dr. Rong Zheng in the computing and software department of the McMaster University, Canada for initial discussion. A PPENDIX -P ROOF OF L EMMA 1: Proof. For each player i, matrix Ri is written as follows: 0 . .. 0 1 i R (t) = 1 0 .. . 0
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
0 .. .
...
0
0
0
0
0
0
...
1
1
−1
1
1
...
...
1
−1
1
1
1 ...
... ...
0 .. .
0 .. .
0 .. .
0 .. .
0 ... .. . ...
0 .. . 0 1 1 0 .. .
...
0
0
0
0
0 ...
0
, (36)
2N ×2N
where all rows are zeros except row i and i + 1. According to Theorem 3 in [27], if matrix A is partitioned row-wise as + A = (A1 :: A2 )T , the matrix M of form M = (A+ 1 A2 ) = + A if the following relationships are satisfied: A+ 1 A2 = 0,
A+ 2 A1 = 0,
(37)
+ ∗ ∗ (A1 A+ 1 ) (A2 A2 ) = 0,
(38)
+ A1 A+ 1 A2 A2 = 0.
(39)
i
Matrix R can have 3 case of row-wise decomposition : 1) non zeros columns are in matrix Ri1 and matrix Ri2 is a 9
[1] N. Ruiz, I. Cobelo, and J. Oyarzabal, “A direct load control model for virtual power plant management,” IEEE Trans. Power Syst., vol. 24, no. 2, pp. 959–966, May 2009. [2] A. Gomes, C. H. Antunes, and A. G. Martins, “A multiple objective approach to direct load control using an interactive evolutionary algorithm,” IEEE Trans. Power Syst., vol. 22, no. 3, pp. 1004-1011, August 2007. [3] C. M. Chu, T. L. Jong, and Y. W. Huang, “A direct load control of air-conditioning loads with thermal comfort control,” in proceedings of IEEE PES Gen. Meet., San Francisco, CA, June 2005. [4] E. Hossain, Z. Han, and V. Poor, Smart Grid Communications and Networking, Cambridge University Press, UK, 2012. [5] A. Mohsenian-Rad and A. Leon-Garcia. “Optimal residential load control with price prediction in real-time electricity pricing environments,” IEEE Transactions on Smart Grid, vol. 1, no. 2, pp. 120-133, September 2010. [6] D. Fudenberg and J. Tirole,Game Theory. MIT Press, 1993. [7] A. C. Tellidou and A. G. Bakirtzis, “Demand response in electricity markets,” in Proc. 15th Int. Conf. Intell. Syst. Appl. Power Syst., pp. 1-6, Curitiba, November 2009. [8] Y. Chen, W. S. Lin, F. Han; Y. H. Yang, Z. Safar, K. J. R. Liu, “A cheatproof game theoretic demand response scheme for smart grids,” 2012 IEEE International Conference on Communications (ICC), pp. 33623366, Ottawa, June 2012. [9] C. Chen, S. Kishore, W. Zhifang, M. Alizadeh, A. Scaglione, “How will demand response aggregators affect electricity markets? - A Cournot game analysis,” 5th International Symposium on Communications Control and Signal Processing (ISCCSP), pp. 1-6, May 2012. [10] S. Dave, M. Sooriyabandara, and L.Y. Zhang, “Application of a gametheoretic energy management algorithm in a hybrid predictive-adaptive scenario,” 2rd IEEE International Conference on Innovative Smart Grid Technologies, Manchester, December 2011. [11] J. Chen, B. Yang, and X. Guan, “Optimal demand response scheduling with Stackelberg game approach under load uncertainty for smart grid,” IEEE Third International Conference on Smart Grid Communications (SmartGridComm), Tainan, Taiwan, pp. 546-551, November 2012. [12] Z. Chen, L. Wu, and Y. Fu, “Real-Time Price-Based Demand Response Management for Residential Appliances via Stochastic Optimization and Robust Optimization,” IEEE Transactions on Smart Grid, vol. 3, issue 4, pp. 1822-1831, December 2012. [13] B. Jiang, and F. Yunsi, “Dynamic Residential Demand Response and Distributed Generation Management in Smart Microgrid with Hierarchical Agents,” Energy Procedia, vol. 12, pp. 76–90, December 2011. [14] A. H. Mohsenian-Rad, V. Wong, J. Jatskevich, R. Schober, and A. LeonGarcia, “Autonomous demand-side management based on gametheoretic energy consumption scheduling for the future smart grid,,” IEEE Trans. Smart Grid, vol. 1, no. 3, pp. 320-331, December 2010. [15] H. K. Nguyen, J. B. Song, and Z. Han, “Demand side management to reduce peak-to- average ratio using game theory in smart grid,” IEEE INFOCOM Workshop on Green Networking and Smart Grids, March 2012. [16] W. Saad, Z. Han, H. V. Poor, and T. Basar, “game-theoretic methods for the smart grid: an overview of microgrid systems, demand-side management, and smart grid communications,” IEEE Signal Processing Magazine, special issue on Signal Processing Techniques for Smart Grid, vol. 29, no. 5, p.p. 86-105, Yongin, South Korea, September 2012. [17] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed. London, U.K.: Academic, 1995.
[18] P. Constantopoulos, F. C. Schweppe, R. C. Larson, “ESTIA: A RealTime Consumer Control Scheme for Space Conditioning Usage under Spot Electricity Pricing,” Computers and Operations Research , vol. 18, no. 8, pp. 751-765, 1991. [19] C. E. Lin and G. L. Viviani,“ Hierarchical economic dispatch for piecewise quadratic cost functions,” IEEE Trans. Power App. Syst., vol.103, no. 6, pp. 11701175, June 1984. [20] N. Li, L. Chen, and S. H. Low, “Optimal demand response based on utility maximization in power networks,” in IEEE Power and Energy Society General Meeting, July 2011. [21] M. G. Crandall, and P. L. Lions, “Viscosity Solutions of Hamilton-Jacobi equations” Transactions of the American Mathematical Society, vol. 277, no. 1, pp. 1-42, May 1983. [22] A. I. Subbotin, “Minimax solutions to the Hamilton Jacobi Equations” Journal of Mathematical Sciences, vol. 103, no. 6, pp. 772-777, March 2001. [23] R. D. Zimmerman, C. E. Murillo-Snchez, and R. J. Thomas, “SteadyState Operations, Planning and Analysis Tools for Power Systems Research and Education,” IEEE Transactions on power systems , vol. 26, no. 1, pp. 12–19, February 2011. [24] A. Mishra, D. Irwin, P. Shenoy, J. Kurose, and T. Zhu. “SmartCharge: Cutting the Electricity Bill in Smart Homes with Energy Storage,” In eEnergy, Proceedings of the 3rd International Conference on Future Energy Systems: Where Energy, Computing and Communication Meet, pp. 21-29, Madrid, May 2012. [25] P. Constantopoulos, F. C. Schweppe, R. C. Larson, “ESTIA: A RealTime Consumer Control Scheme for Space Conditioning Usage under Spot Electricity Pricing,” Computers and Operations Research, vol. 18, no.8, pp. 751-765, June 1991. [26] http://www.amigoenergy.com. [27] J.K. Baksalary, O.M. Baksalary, “Particular formulae for the MoorePenrose inverse of a columnwise partitioned matrix,” Linear Algebra and Its Applications no. 421, pp.16-23, 2007.
10
