Distributed Iterative Regularization Algorithms for ... - Semantic Scholar
I. I NTRODUCTION Consider an N −person Nash game in which the ith player solves Ag(x−i )
minimize
fi (xi ; x−i )
subject to
xi ∈ Ki
for i = 1, . . . , N where x−i = (xj )j6=i , Ki ⊆ Rni is a closed PN and convex set and i=1 ni = n. Additionally, we assume that for i = 1, . . . , N , the function fi is a differentiable real-valued function given by fi : Rn → R and convex Q in xi for all x−i ∈ j6=i Kj . A Nash equilibrium of the aforementioned noncooperative game is given by a tuple {x∗i }N i=1 , where x∗i ∈ SOL(Ag(x∗−i )) for i = 1, . . . , N and SOL(Ag) denotes the set of solutions to problem (Ag). Throughout this paper, we refer to this canonical Nash game by G. The convexity of the objectives and the associated strategy sets allow us to claim that Department of Industrial and Enterprise Systems Engineering, University of Illinois, Urbana IL 61801, Email: {akannan,udaybag}@illinois.edu. This work has been partially supported by NSF award CCF-0728863 and DOE award DE-SC0003879.
that the first-order equilibrium conditions are necessary and sufficient. In fact, these conditions can be shown to be equivalent to a scalar variational inequality VI(K, F ) (see result from [5, Ch. 1]) where ∇x1 f1 (x) N Y . . Ki . F (x) , and K , . ∇xN fN (x)
i=1
Recall that VI(K, F ) is a problem requiring an x ∈ K such that F (x)T (y − x) ≥ 0, ∀y ∈ K. We assume throughout this paper that the mapping F : Rn → Rn is a single-valued mapping possessing a monotonicity property over K namely: (F (x) − F (y))T (x − y) ≥ 0 for all x, y ∈ K. We refer to the resulting class of Nash games as monotone Nash games. Much research has been carried out on the development of algorithms for monotone variational inequalities, amongst these being projection-based methods, interior-point algorithms, etc (see [5, Ch. 12]). Of these, projection-based schemes are natural candidates for employment within a distributed framework. Yet, by themselves, standard projection methods require a degree of well-posedness in order to claim global convergence. In particular, if F is either strongly monotone or co-coercive, we may develop a single timescale fixed steplength scheme under the caveat that the steplength is sufficiently small. A single timescale scheme refers to one where each iterate is obtained by a single gradient or projection step. In particular, if F is strongly monotone over K and Lipschitz continuous, then such fixed steplength schemes are directly available. However, if F is merely monotone, generally such an avenue is unavailable. Several alternatives may be examined in the absence of strong monotonicity. A set of classical techniques reside in the realm of regularization and proximal-point methods and requires solving a sequence of well-posed problems, each of which might require a distributed iterative process in itself. This is effectively a two timescale method where the regularization/proximal method makes changes at a slower timescale while solutions to the regularized problems in an exact or inexact form are found at a faster timescale. Our motivation lies in developing distributed single timescale extensions of standard regularization/proximal methods with appropriate convergence guarantees. In general, these schemes require updating the regularization parameters at every iteration, rather than when an acceptable subproblem solution is avail-
able, a property that leads to the schemes being referred to as iterative regularization techniques. Our work is motivated by a host of settings where online distributed schemes assume relevance. Examples of these include communications [4], [10], bandwidth and spectrum allocation [2], [3] and optical networks [7], [8]. Of note is recent work of Pavel and her coauthors [7] where an extragradient scheme [5] in a deterministic regime, capable of accommodating monotone Nash games. Also of interest is recent work that examines best response schemes in the context of monotone Nash games [10]. The key contributions of this paper lie in the introduction and analysis of several iterative regularization techniques. The first of these extends a standard Tikhonov regularization method to the single-timescale regime by requiring that the regularization parameter be updated at every iteration. Correspondingly, in the context of proximal-point methods, we suggest that the centering parameter around which the prox-term is constructed is also updated at every iteration. A third scheme is presented where the weighting parameter in the prox-term is also updated at every iteration. In the instance of Tikhonov, the scheme accommodates merely monotone mappings while the proximal-point based methods currently require strict monotonicity. This paper is divided into several sections. In section II, we provide a brief background to the methods of interest, namely Tikhonov regularization methods and proximal-point approaches. In section III, we extend Tikhonov regularization methods to the distributed single timescale regime in the context of symmetric and general monotone Nash games. Section IV presents a single timescale iterative proximal point method while section V introduces a modified iterative proximal scheme. In section VI, we present some numerical results and provide some concluding remarks in section VII. II. P RELIMINARIES Our analysis is restricted to Nash games that lead to monotone variational inequalities, a class of games that takes on the name monotone Nash games. A host of schemes exist for the solution of variational inequalities, many of which do not even require monotonicity. Our goal, however, lies in developing distributed schemes implementable in networked settings. Since, strong monotonicity of the mapping allows us to directly cnstruct precisely such a class of schemes, this forms our starting point. In the absence of strong monotonicity however, a direct application of direct projection schemes does not guarantee a contraction and thereby convergence [5]. A possible avenue for alleviating the challenge is through regularization methods, a set of techniques that address this ill-posedness by sequentially solving a set of strongly monotone problems. One such technique is an exact Tikhonov regularization method. Let F k , F (z) + k z, k+1
k
k
(1)
where z = SOL(K, F ) and denotes the regularization at the kth iteration. suitable conditions (see [5, Under Ch.12]) the sequence z k converges to z ∗ as k → 0. The
subproblems are strongly monotone variational inequalities and can be solved by a host of iterative methods. Note that inexact solutions of such problems also leads to convergent schemes [5, Ch. 12]. An alternative scheme is one that maintains a regularization based on the change in consecutive iterates via a proximal term. Such methods are also referred to as heavyball methods [9]. Given a scalar θ > 0, such a scheme employs an F k be redefined as F k , F (z) + θ(z − z k−1 ), k+1
(2)
k
where z = SOL(K, F ). This scheme is guaranteed to converge for monotone variational inequalities (see [1], [5]). A key shortcoming, however, is that both schemes require the solution of a sequence of problems, leading to a natively two timescale scheme. Single timescale schemes have several advantages. First, they are far easier to implement since the complexity of the scheme is restricted to solving the projection problem. Second, online implementations that require coordination in a networked setting are far easier to manage, making them significantly attractive over schemes that require solving complex subproblem prior to updating their decisions. Third, these schemes are easily distributed providing an avenue for solving truly large-scale networked game-theoretic problems. With this being the major motivation, we provide a unified framework for stating three different single timescale schemes namely the iterative Tikhonov scheme, the iterative proximal scheme and the modified iterative proximal scheme. Given a game G, the general form of the single timescale scheme is as follows: k ) + θ1k zik − θ2k zik−1 . (3) zik+1 = ΠKi zik − γk F (zik , z−i Based on the choices of the parameter sequences {θ1k } and {θ2k }, we have the following schemes: a) Case 1: If, θ1k = k → 0 and θ2k = 0, then the scheme is referred to as the iterative Tikhonov regularization (ITR) scheme. b) Case 2: If θ1k = θ2k = θ is fixed, then the scheme is referred to as the iterative proximal point (IPP) scheme. c) Case 3: If θ1k = θ2k = γck where c ∈ (0, 1), then the scheme is referred to as the modified iterative proximal point (MIPP) scheme. III. I TERATIVE T IKHONOV SCHEMES In standard Tikhonov regularization schemes, one constructs a sequence of exact (or inexact) solutions to wellposed regularized problems and the regularization parameter is driven to zero at the slower timescale. In contrast, we consider a class of iterative Tikhonov schemes in which the steplength and regularization parameter are changed at the same rates. We proceed to show that this scheme is indeed convergent. The convergence statement for general monotone mappings also appears in [6] but does so without a proof and employs slightly different assumptions. Our proof is original and was presented in a shortened form in past work by the second author [11]. Here, we restate the proofs with some
modifications with the intent of generalizing it to partially coordinated settings. Notably, the result for symmetric Nash games is inspired by a result for optimization problems stated in [9]. The following Lemmas from [9] are employed in developing our convergence theory. Lemma 1: Let uk+1 ≤ qk uk + αk , 0 ≤ qk < 1, αk ≥ 0 and ∞ X αk (1 − qk ) = ∞, → 0, k → ∞. 1 − qk k=0
Then limk→∞ uk ≤ 0 and if uk > 0, then limk→∞ uk = 0. Lemma 2: Let uk+1 ≤ (1 + vk )uk + pk , uk , vk , pk ≥ 0 and ∞ ∞ X X vk < ∞, pk < ∞, k → ∞. k=0
k=0
Then limk→∞ uk = u ¯≥0 Our proof of convergence relies on relating the iterates of the proposed ITR scheme to that of the original Tikhonov scheme. The following Lemma is reproduced from [11] provides a bound between consecutive iteratives of the standard Tikhonov scheme. Lemma 3: Let the mapping ∇F be monotone and suppose SOL(K, F ) be nonempty and bounded. Consider the standard exact tikhonov scheme defined by (1). If M := kw∗ k2 , then M (k−1 − k ) ky k − y k−1 k ≤ . k Proof: Omitted (See [11]). In Section III-A, we prove that the ITR scheme can be shown to be convergent. When the Jacobian ∇F is assumed to be symmetric, a fixed steplength scheme is available even in monotone regimes . When ∇F is not necessarily symmetric, we present a diminishing steplength scheme in Section III-B.
sequence of iterates generated by the ITR scheme be denoted by {wk }. Then kwk+1 − y k k ≤ qk kwk − y k k + αk , where qk = (1 − γk ) and αk = qk M
k−1 − k . k
In addition, limk→∞ kwk+1 − y k k = 0. Proof: Omitted (See [11]). Before proceeding, its worthwhile examining whether (A6) does have a feasible choice of sequences. We provide a simple result from [11] that provides a set of acceptable choices. Lemma 4: Consider assumption (A6). Then the update scheme k = k −α satisfies (4) in (A6) where α ∈ (0, 1) for all k. Proof: Omitted (see [11]). B. General monotone Nash games When considering games with mappings that have general Jacobians, it is not possible to obtain a bound by means of the mean value theorem. Therefore, we follow a different methodology, namely a diminishing step size to ensure a convergent sequence. Theorem 2: Suppose the G has a nonempty bounded set of equilibria. Let K be convex and let F be monotone on K. Let assumption (A6) hold. Then wk → w∗ as k → ∞, where wk is obtained via the iterative Tikhonov scheme and w∗ is the least-norm solution of VI(K, F ). Proof: Through the use of Lemma 3 and Theorem. 1, we obtain the following sequence of relationships: kwk+1 − y k k ≤ qk kwk − y k k ≤ qk kwk − y k−1 k + qk ky k − y k−1 k
A. Symmetric monotone Nash games In this section we consider the class of symmetric monotone Nash games; a class of monotone games where the Jacobian ∇F is symmetric. The convergence of the ITR scheme for this case is discussed in the following theorem, sourced from [11]. Moreover, the mapping, the regularization parameter {k } and the steplength sequence {γk } are assumed to satisfy the following assumptions based on whether the mapping has a symmetric Jacobian or not. Assumption 1: (A6) The mapping F (x) is Lipschitz continuous with constant L. The parameter k Pregularization ∞ k k and steplength γk satisfy γ = ∞, k+1 ≤ k=1 P ∞ k 2 k , ∀k, γ ≤ γ ∀k, lim γ / = 0, k→∞ k k=0 γk < P∞ k+1 k 2 k ∞, k=0 (γk ) < ∞ and k−1 − k = 0. (4) k→∞ γk (k )2 Proposition 1: Suppose the G has a nonempty bounded set of equilibria. Let K be convex and let F be monotone on K. In addition, let ∇F be symmetric and uniformly bounded by a constant D over x ∈ K. Suppose (A6) holds and the lim
≤ qk kwk − y k−1 k + qk M
(k−1 − k ) , k
where the second inequality is a consequence of the triangle inequality and the third inequality follows from Lemma 3. By the triangle inequality, kwk+1 − w∗ k can be bounded by terms 1 and 2: kwk+1 − w∗ k ≤ kwk+1 − y k k + ky k − w∗ k . | {z } | {z } term 1
term 2
Of these, term 2 converges to zero from the convergence statement of Tikhonov regularization methods. It suffices to show that term 1 converges to zero as k → ∞ which follows as shown next. By using the non-expansivity of the Euclidean projector, kwk+1 − y k k2 is given by kwk+1 − y k k2 = kΠK wk − γk (F (wk ) + k wk ) − ΠK y k − γk (F (y k ) + k y k ) k2 ≤ k wk − γk (F (wk ) + k wk ) − y k − γk (F (y k ) + k y k ) k2 .
This expression can be further simplified as
This expression can be simplified as follows:
k(1 − γk k )(wk − y k ) − γk (F (wk ) − F (y k ))k2 = (1 − γk k )2 kwk − y k k2 + γk2 kF (wk ) − F (y k )k2 − 2γk (1 − γk k )(wk − y k )T (F (wk ) − F (y k )) ≤ (1 − 2γk k + γk2 (L2 + 2k ))kwk − y k k2 , where the last inequality follows from γk k ≤ 1 and the monotonicity of F (x) over K. If Lemma 1 can indeed be invoked then it follows that kwk+1 − y k k → 0 as k → ∞. The remainder of the proof shows that the conditions for employing Lemma 1pdo hold. Suppose qk := (1 − 2γk k + γk2 (L2 + 2k ). Invoking Lemma 1 requires showing that ∞ X
k−1
(1−qk ) = ∞
lim
and
k→∞
k=0
k
− ) = 0. k
qk ( M 1 − qk
It is easily seen that ∞ X
(1 − qk ) =
k=0
∞ X 1 − q2 k
k=0
1 + qk
= >
∞ X 2γk k − γ 2 (L2 + 2 ) k
k=0 ∞ X
k
1 + qk
This limit is now of αthe form limk→∞ (h1 (k)−h2 (k)) where k2 β h1 (k) = k k−1 , h2 (k) = k β k α , limk→∞ h1 (k) = ∞ and limk→∞ h2 (k) = ∞. Furthermore, this limit can be expressed as 1/h2 (k) − 1/h1 (k) , k→∞ 1/(h1 (k)h2 (k))
lim (h1 (k) − h2 (k)) = lim
k→∞
which is of a form that allows for the application of L’Hopital’s rule. This leads to the following simplification α α−1 k −1 k − 1 α k−1 k−1 (k−1)2 lim = lim k→∞ k −(α+β) k→∞ (−(α + β))k −(α+β+1) α+1 k α k α+β−1 = 0, = lim k→∞ α + β k−1 where the last equality comes from the fact that α+β−1 < 0.
(2γk k − γk2 (L2 + 2k )) = ∞,
k=0
where the inequality follows qk < 1 and the final P∞ from k equality follows from γ = ∞ and the square k k=0 summability of γk k and γ k . The second requirement follows by observing that (k−1 − k ) qk qk (1 + qk ) (k−1 − k ) M M = 1 − qk k 1 − qk2 k k−1 ( − k ) qk (1 + qk ) M = 2γk k − γk2 (L2 + 2k ) k k−1 qk (1 + qk ) ( − k ) = M . 2 − γkk (L2 + 2k ) γk (k )2 | {z } | {z } T erm 1
k−1 − k L2 (k − 1)−α − k −α = lim k→∞ γk (k )2 k→∞ δ k −2(α+β) α 2 L 1 α+β = lim − k α+β ). (k k→∞ δ 1 − 1/k lim
T erm 2
k
Since γk , → 0, it follows that qk → 1. Since γk /k → 0, Term 1 tends to 1 as k → ∞. By assumption, Term 2 tends to zero as k → ∞. Again, the following result shows that a feasible choice of steplength and regularization parameter sequences do indeed exist for satisfying (A6), under a slightly different set of assumptions from [11]. Lemma 5: Consider assumption (A6). Then the update scheme k = k −α and γ k = k −β satisfies (4) in (A6) where 1 2 < α + β < 1 and β > α for all k. Proof: Note that by the choice of γk and k , we 0 immediately have that γk
α. We also have that k=0 γk = ∞, k=0 = ∞ and limk→∞ γk /k = 0, where the last result follows from β > α. The square summability of γk k is a consequence of 2(α + β) > 1. It remains to show that the limit given by (4) does indeed hold for this choice of {k } and {γk }: (k−1 − k ) lim = 0. k→∞ γk (k )2
IV. I TERATIVE PROXIMAL POINT SCHEME In this section, we consider an alternate technique for alleviating the absence of strong monotonicity. This method uses a proximal term of the form θ(z −z k−1 ) rather than k z in modifying the map. Consequently, when such a method is applied to a variational inequality VI(K, F ), a sequence of iterates is constructed, each of which requires the solution of a modified strongly modified problem VI(K, F k )), where F k , F (z) + θ(z − z k−1 ). The convergence of the proximal-point algorithm is shown to hold under an assumption of monotonicity of the original mapping and for a positive θ. Note that the solution of each subproblem, given by z k = (SOL(K, F k )), is given by the solution of the fixed-point problem z k = ΠK (z k − γ F (z k ) + θ(z k − z k−1 ) . Under assumptions of convexity of the set K and monotonicty of F on K, the convergence of the standard proximalpoint algorithm has been established in [1], [5]. But, this scheme suffers from a key drawback; it is essentially a two timescale scheme requiring the solution of a variational inequality at every iteration. In the spirit of the iterative Tikhonov regularization scheme, we present a single timescale iterative proximal point (IPP) method. In a game-theoretic generalization of this scheme, a projection step using the deviation between the k th and k − 1th iterates yields the k + 1th iterate and is formally stated as zik+1 = ΠKi zik − γk (Fi (z k ) + θ(zik − zik−1 )) , (5) for i = 1, . . . , N. Next, under an assumption of strict monotonicity of F (x) and the boundedness of K, we establish the global convergence of the IPP scheme.
Theorem 3: Consider the game G and assume that K is a compact and convex set and F is a continuous and strictly monotone map on K. Let {zk } denote the set of iterates proximal scheme (5). Let P∞ defined by the P∞iterative 2 γ = ∞ and γ < ∞. Then limk→∞ zk = w∗ . k k=1 k=1 k Proof: We begin by expanding kz k+1 − w∗ k and by using the non-expansivity property of projection.
kz k+1 − w∗ k2 : kz k+1 − w∗ k2 ≤ (1 + γk2 L2 )kz k − w∗ k2 2 + (γk θ)2 γk−1 (β + θC)2 + 2γk γk−1 θC(1 + γk L)(β + θC)
≤ (1 + γk2 L2 )kz k − w∗ k2 | {z } +
kz k+1 − w∗ k2 = kΠK (z k − γk (F (z k ) + θ(z k − z k−1 ))) − ΠK (w∗ − γk F (w∗ ))k2 ≤ k(z k − w∗ ) − γk (F (z k ) − F (w∗ )) − γk θ(z k − z k−1 )k2 .
,vk 2 2 (γk θ) γk (β
|
,pk
The above sequence can be compactly represented as the recursive sequence uk+1 ≤ (1 + vk )uk + pk , where ∞ X
Expanding the right hand side,
k=0
kz k+1 − w∗ k2 ≤ kz k − w∗ k2 + (γk L)2 kz k − w∗ k2 + (γk θ)2 kz k − z k−1 k2 − 2γk (z k − w∗ )T (F (z k ) − F (w∗ )) − 2γk θ(z k − z k−1 )T z k − w∗ − γk (F (z k ) − F (w∗ )) . Using Lipschitz and monotonicity properties of F (x), we have kz k+1 − w∗ k2 ≤ (1 + γk2 L2 )kz k − w∗ k2 + (γk θ)2 kz k − z k−1 k2 −2γk θ(z k − z k−1 )T (z k − w∗ ) − γk (F (z k ) − F (w∗ )) . | {z } Term 1
Term 1 can bounded from above by the use of the CauchySchwartz inequality, the boundedness of the iterates, namely kz k − w∗ k ≤ C, and the Lipschitz continuity of F , as shown next.
+ θC)2 + 2γk2 θC(1 + γk L)(β + θC) . {z }
vk = L2
∞ X
∞ X
γk2 < ∞,
k=0
pk < ∞,
k=0
the latter a consequence of the square summability of γk . It follows from Lemma 2 that uk → u¯ ≥ 0. It remains to show that u¯ = 0. Recall that kz k+1 − w∗ k2 is bounded as per the following expression: kz k+1 − w∗ k2 ≤ (1 + vk )kz k − w∗ k2 + pk − 2γk (z k − w∗ )T (F (z k ) − F (w∗ )) Suppose u¯ > 0. It follows that along every subsequence, we 0 have that µk = (zk − w∗ )T (F (zk ) − w∗ ) ≥ µ > 0, ∀k. This is a consequence of the strict monotonicity of F whereby if (F (z k ) − F (w∗ ))T (z k − w∗ ) → 0 if z k → w∗ . Since u¯ > 0, it follows that kz k − w∗ k2 → u¯ > 0. Then by summing over all k, we obtain lim kz k+1 − w∗ k2 ≤ kz 0 − w∗ k2 +
k→∞
γk2 L2 kz k − w∗ k2
k=0
−2
kz k+1 − w∗ k2
∞ X
∞ X
γk µk +
k=0
≤ (1 + γk2 L2 )kz k − w∗ k2 + (γk θ)2 kz k − z k−1 k2
∞ X
pk .
k=0
Since vk and pk are summable and µk ≥ µ0 > 0 for all k, + 2γk θk(z k − z k−1 )k k(z k − w∗ )k + γk k(F (z k ) − F (w∗ ))k we have that ≤ (1 + γk2 L2 )kz k − w∗ k2 + (γk θ)2 kz k − z k−1 k2 ∞ X k+1 ∗ 2 0 ∗ 2 k k−1 2 k k−1 lim kz − w k ≤ kz − w k + γk2 L2 kz k − w∗ k2 + 2γk θCk(z − z )k + 2γk θLCk(z − z )k. k→∞
Next, we derive a bound on kz k − z k−1 k by leveraging the non-expansivity of the Euclidean projector. kz k − z k−1 k = kΠK (z − ΠK (z ≤ k(z
k−1
k−1
k−1
−2
γk µk +
k=0
∞ X
pk
k=0 ∞ X
≤ kz 0 − w∗ k2 +
− γk−1 (F (zk−1 ) + θ(z
k−1
−z
k−2
− 2µ0 k−1
−z
k−2
))) − (z
γk2 L2 kz k − w∗ k2
k=0
))))
)k
− γk−1 (F (zk−1 ) + θ(z
k=0 ∞ X
k−1
)k
= k − γk−1 (F (zk−1 ) + θ(z k−1 − z k−2 )))k. It follows from the boundedness of K and the continuity of F (z), that there exists a β > 0 such that kF (z)k ≤ β for all z ∈ K, implying that kz k − z k−1 k ≤ γk−1 (β + θC). The bound on kz k − z k−1 k allows us to derive an upper bound
∞ X k=0
γk +
∞ X
pk ≤ −∞,
k=0
P∞ 2 where that k=0 γk < P∞the latter follows from observing ∞, k=0 γk = ∞ and kzk − w∗ k ≤ C. But this is a contradiction, implying that along some subsequence, we k ∗ 2 have that µk → k0 and lim inf k→∞ kz − w k = 0. But we know that z has a limit point and that the sequence zk converges. Therefore, we have that limk→∞ zk = w∗ .
V. M ODIFIED ITERATIVE PROXIMAL POINT SCHEME In this section, we consider a modified form of the iterative proximal-point scheme. This modification is motivated by observing that the proximal term at the kth iterate, given by θ(z k − z k−1 ), is O(γk−1 ), a consequence of the bound developed in the previous result. Therefore, could one develop an iterative proximal-point method where the proximal term was less dependent on the steplength?. Furthermore, it would be expected that this proximal term would converge to zero at a slower rate. Yet, would we find that the behavior of the trajectory was indeed smoother and this proximal term would provide better damping. We consider a modified IPP scheme in which the ith player takes a step given by zik+1 = ΠKi zik − γk (Fi (z k ) + θk (zik − zik−1 )) , (6) for i = 1, . . . , N , where θk = c/γk and c ∈ (0, 1). It follows that this scheme can be effectively stated as zik+1 = ΠKi zik − γk (Fi (z k ) + θk (zik − zik−1 )) = ΠKi zik − γk Fi (z k ) − c(zik − zik−1 )) = ΠKi (1 − c)zik + czik−1 − γk Fi (z k )) . Essentially, a projection step is carried out using a convex combination of zik and zik−1 . Note that even in the standard iterative proximal-point scheme, such a combination is used; however, in that setting, combination is of the form (1 − γk θ)zik + γk θzik−1 and as one proceeds, γk → 0 and one places less and less emphasis on the past. In this setting, we use a fixed convex combination, specified by the parameter c ∈ (0, 1). Prior to proving our main convergence statement, we prove an intermediate result. Lemma 6: Suppose uk ≤ cuk−1 + γk−1 β where c ∈ (0, 1), {γk } > 0 is a decreasing bounded sequence with P∞ k=0 Pγ∞k < ∞ and 0 ≤ uk < ∞ for all k. Then, we have that k=1 γk uk < ∞. Proof: By definition, we have that uk ≤ cuk−1 +γk−1 β for k = 1, . . . ,. Multiplying this expression by γk−1 and summing over k, we obtain k X
γj−1 uj ≤ c
j=1
k−1 X
γj uj + β
j=0
k X
since
P∞
j=0
Proposition 4: Consider a game G and and assume that K is a compact and convex set and F is a continuous and strictly monotone map on K. Let {zk } denote the set of iterates proximal scheme (6). Let P∞ defined by the P∞iterative 2 γ = ∞ and γ < ∞. In addition let θk = γck k k=1 k=1 k where c ∈ (0, 1) for k ≥ 0. Then limk→∞ zk = w∗ . Proof: We begin by observing that kz k − z k−1 k can now be bounded as follows: kzk − zk−1 k ≤ kΠK (zk−1 − γk−1 (F (zk−1 ) + θk (zk−1 − zk−2 ))) − ΠK (zk−1 )k ≤ k − γk−1 (F (zk−1 ) + θk (zk−1 − zk−2 ))k ≤ γk−1 kF (zk )k + γk−1 θk−1 kzk−1 − zk−2 k = γk−1 kF (zk )k + ckzk−1 − zk−2 k. The above sequence is of the form uk+1 = qk uk +αk , where qk = γk−1 θk−1 = c and αk = γk−1 β where kF (z k )k ≤ β. Therefore, we have that ∞ X
αk αk = lim = 0. k→∞ (1 − c) 1 − qk k=1 Therefore from Lemma 1, the sequence z k converges and (1−qk ) = ∞
γj uj ≤
j=1
=⇒ (1 − c)
k X
γj−1 uj ≤ c
γj2 .
k−1 X
γj uj + β
j=0
j=1
γj uj + γk uk ≤ cγ0 u0 + β
j=1
lim kz k − z k−1 k = 0.
Since w∗ is bounded and fixed, it is clear that, kz k − w∗ k converges to u¯ ≥ 0. It suffices to show that u¯ ≡ 0. We proceed by contradiction and assume that u¯ > 0. We begin by recalling the definition of iterates and leveraging properties of the projection operator, we have kzk+1 − w∗ k2 ≤ kzk − γk (F (zk ) + θk (zk − zk−1 )) − (w∗ − γk F (w∗ ))k2 = k(1 − γk θk )(zk − w∗ ) − γk (F (zk ) − F (w∗ ))
By expanding the expression on the right, we have (1 − γk θk )2 k(zk − w∗ )k2 | {z }
k−1 X
γj2
k−1 X
k X γj uj + lim γk uk ≤ cγ0 u0 + β lim γj2 k→∞ k→∞ k→∞ | {z } j=0 j=0
=⇒ lim
k→∞
=0 k−1 X
γj uj < ∞,
j=0
+ γk2 θk2 k(zk−1 − w∗ )k2 | {z } Term 2
γj2 .
j=0
k−1 X
+
Term 1 2 γk k(F (zk ) − F (w∗ ))k2
j=0
By taking limits, it follows that (1 − c) lim
lim
k→∞
+ γk θk (zk−1 − w∗ )k2 .
j=0
k−1 X
and
k→∞
But γk ≤ γk−1 for all k, implying that k X
γj2 < ∞, limk→∞ γk = 0 and uk is bounded.
+ 2γk θk (1 − γk θk )(zk − w∗ )T (zk−1 − w∗ ) | {z } Term 3
−2γk2 θk (F (zk ) − F (w∗ ))T (zk−1 − w∗ ) | {z } Term 4
−2γk (1 − γk θk )(zk − w∗ )T (F (zk ) − F (w∗ )) . | {z } Term 5
By using Cauchy-Schwartz on term 3, and subsequently combining with terms 1 and 2, leads to term 6 below. Additionally, terms 4 and 5 when added together, along with the
P∞ P∞ that k=0 dk < ∞, k=0 γk µk = ∞ and the boundedness of z k , it emerges that
application of Cauchy-Schwartz, lead to the corresponding terms below. kzk+1 − w∗ k2 ≤ ((1 − γk θk )kzk − w∗ k + γk θk kzk−1 − w∗ k) | {z }
∞ X
− 2γk (zk − w∗ )T (F (zk ) − F (w∗ ))
k=1
2
Term 6
2
(c(1 − c))2 ((kzk − w∗ k − kzk−1 − w∗ k) ≤ −∞,
a contradiction to the nonnegativity of the left-hand side.
+ 2γk2 θk kF (zk ) − F (w∗ )kkzk − zk−1 k + γk2 kF (zk ) − F (w∗ )k2 . | {z } Therefore along some subsequence, we have that µk → 0 k ∗ 2 k
and lim inf k→∞ kz −w k = 0. But we know that z has a limit point and that the sequence zk converges. Therefore, we have that limk→∞ zk = w∗ .
,dk
Consequently, kz k+1 − w∗ k2 can be expressed as 2
kz k+1 − w∗ k2 ≤ ((1 − γk θk )kzk − w∗ k + γk θk kzk−1 − w∗ k)
VI. N UMERICAL EXPERIMENTS
− 2γk (zk − w∗ )T (F (zk ) − F (w∗ )) + dk .
The performance of the proposed schemes is tested on a networked Nash-Cournot game. A set of firms denoted by J is assumed to compete over a network of nodes, denoted by N . We denote the production and sales of firm j ∈ J at node i by yij and sij , respectively. Furthermore, the cost of production faced by firm j at node i is assumed to be linear 2 Finally, the nodal price pi at node i is kz k+1 − w∗ k2 ≤ ((1 − c)kzk − w∗ k + ckzk−1 − w∗ k) + dk and is denoted by Cij .P given by p , a − b ∗ T ∗ i i i j∈J sij . For purposes of simplicity, − 2γk (zk − w ) (F (zk ) − F (w )). we assume that the transmission cost is zero and generation is bounded by capacity constraints. In addition, total sales By summing over k, we have across all nodes should be equal to the total generation across K K X X all2 nodes. We specify firm j’s optimization problem by kz k+1 − w∗ k2 ≤ ((1 − c)kzk − w∗ k + ckzk−1 − w∗ k) k=1 k=1 X X K K (ai − bi X X maximize sij )sij − Cij (yij ) + dk − 2 γk (zk − w∗ )T (F (zk ) − F (w∗ )). i∈N j∈J k=1 k=1 yX ij ≤ capijX In the expression above, we observe that the coefficient of y = s ij ij subject to , Kj kz k − w∗ k2 for 2 ≤ k ≤ K − 2 is given by (1 − (1 − c)2 − i∈N i∈N yij , sij ≥ 0 c2 ) = 2c(1 − c). It follows that the inequality above can be expressed as The compactness of Kj implies that the game admits K−1 a solution. Let the variational formulation X QJ of the game 2 c(1 − c) (kzk−1 − w∗ k − kzk − w∗ k) be denoted by VI(K, F ) where K = j=1 Kj , F = k=1 T T T F1 . . . FN , − c(1 − c) kz 0 − w∗ k2 + 2kz 0 − w∗ k2 0 X 1 k k−1 2 2 k. It can be observed that dk ≤ γP k β + 2cβγk kz − z ∞ k k−1 From Lemma 6, it is clear that γ kz − z k < ∞. P∞ k=1 k This allows us to claim that k=1 dk < ∞. By recalling that γk θk = c, the error kz k+1 − w∗ k2 can be further bounded
+ (1 − c)kz K−1 − w∗ k2 + kz K − w∗ k2 ≤
K X
dk − 2
k=1
K X
γk (zk − w∗ )T (F (zk ) − F (w∗ )).
k=1
bi si1 + bi si1 − ai 0 1 B j∈J C Ci1 B C B C y B . C .. Fis = B C , Fi = @ .. A , . BX C @ A CiJ siJ + bi siJ − ai j∈J
Taking limits, we have ∞ X
0
Fi =
2
c(1 − c) (kzk − w∗ k − kzk−1 − w∗ k)
k=1
− c(1 − c) kz 0 − w∗ k2 + 2kz 0 − w∗ k2 k
∗ 2
+ lim kz − w k + lim (1 − c)kz ≤
Fis Fiy
„
k→∞ ∞ X
k→∞
k−1
∗ 2
−w k
(dk − γk µk ).
k=1
Since u¯ > 0, it follows that along every subsequence, we 0 have that µk = 2(zk −w∗ )T (F (zk )−w∗ ) ≥ µ > 0, ∀k. This is a consequence of the strict monotonicity of F whereby if (F (z k ) − F (w∗ ))T (z k − w∗ ) → 0 if z k → w∗ . By noting
«
∇F1 B , ∇F = @ ... 0
... .. . ...
1 0 „ .. C , ∇F = Ai i . A 0 ∇FN
« 0 0
and Ai = bi (I + eeT ). Clearly, Ai is positive definite for bi > 0 and the mapping is monotone and symmetric. In our test case, we consider a four node setting with the intercepts and slopes are nonnegative and drawn from a normal distribution N (250, 20) and N (1, 0.05) respectively. The capacities and costs are also assumed to be nonnegative and drawn from N (60, 15) and N (25, 10) respectively. Insights for ITR: For iterative Tikhonov schemes, the step length was specified as k −β where β = 0.51 while 0 = 1e−4 . Furthermore, suppose z 0 ≡ 0 and we terminate
TABLE III
TABLE I S YMMETRIC AND A SYMMETRIC ITR SCHEMES Firms 8 10 12 14
ITR-asymmetric (k = k−α ) α = 0.35 α = 0.40 α = 0.45 15612 15598 15589 929 929 929 11559 11536 11524 5881 5881 5881
IPP AND MIPP SCHEMES
ITR-symmetric (k = k−α ) α = 0.45 α = 0.6 α = 0.75 2985 2973 2968 628 628 628 2957 2587 2571 1746 1746 1746
Firms 8 10 12 14
θ =2 1575 212 836 637
IPP (θ) θ =4 1915 299 1042 865
θ =6 2213 418 1276 1169
MIPP (γk = k−β ) β =0.54 β =0.58 β =0.62 3398 5073 8003 392 438 535 1333 2146 3229 403 1205 2714
TABLE II M ULTI - STEP ITR Firms 8 10 12 14
r=1 15650 929 11692 5881
ITR-Asymmetric r=2 r=3 r=4 5105 2668 1718 296 152 96 4988 7741 10884 1890 983 630
r=5 1263 67 15027 2159
if the fixed-point relationship is satisfied within a tolerance of δ = 1e−6 . Table I shows the number of iterations for the symmetric and asymmetric ITR schemes. Symmetric approaches with fixed steplengths perform significantly better than asymmetric versions. It is also apparent that both schemes perform marginally better as the regularization parameter is driven to zero at a faster rate. Insights for multi-step ITR: We tested a variant of the suggested ITR, where instead of a single projection step, a fixed number of projection steps are taken at every iteration. In other words at every step, the iterate is closer to the true Tikhonov iterate. In testing this scheme, we assumed that the steplength and regularization were changed at the rate k −β and k −α respectively, where β = 0.51 and α = 0.25. Table II reports the number of outer iterations (steps) taken with increasing number of fixed inner projection steps (r). It emerges that ITR often performs better (in terms of overall complexity) when there are multiple inner projection steps. Yet, beyond a certain number of steps, the complexity of the scheme grows. We observe that as r grows, the scheme tends to a best-response method. Notably, standard Tikhonov schemes require that as one proceeds, the regularized problems are solved with increasing exactness. The multi-step ITR is distinct in that the complexity of each major iteration is identical. Furthermore, communication overhead often reduces when r grows upto a certain level. Insights for IPP/MIPP: In testing the IPP and MIPP schemes, we assumed that the steplength was changed at the rate k −β where β = 0.51. Table III reports the number of iterations taken by IPP with increasing θ and the number of iterations taken by MIPP with increasing β (c = 0.5). First, it is evident that the performance of the scheme improves with smaller values of θ. Additionally, as the steplength is driven to zero at a faster rate, there is a distinct deterioration of performance of MIPP. As a final remark, it appears that for the test problems in question, the proximal schemes perform significantly better than ITR schemes. VII. C ONCLUDING REMARKS In this paper, we have considered the development of single timescale distributed techniques for the solution of monotone Nash games over continuous strategy sets. When the maps associated with the Nash games admit a strong
monotonicity property, then a constant steplength distributed scheme is immediately available. However, we consider regimes where the mapping is monotone or strictly monotone and investigate the use of regularization techniques. Generally, such approaches require the solution of a sequence of well-posed single-valued problems; a direct employment of such schemes would lead to a natively two-timescale framework while our goal lies in a single timescale method. Consequently, we turn to an iterative regularization framework in which the regularization parameter is updated at every iterate, rather than when an exact or approximate solution of the subproblem is available. We present three such schemes and their associated convergence theory under appropriate assumptions. The first of these is an iterative Tikhonov method that relies on the solution of regularized problems that can accommodate merely monotone mappings. The second schemes is an iterative proximal-point method whose convergence requires both compactness of the set and strict monotonicity of the mapping. The third scheme is a modified proximal-point scheme is one where the weight of the proximal-point term is updated as well. Finally, the performance of the schemes is examined on a networked Nash-Cournot game. R EFERENCES [1] E. Allevi, A. Gnudi, and I.V. Konnov, The Proximal Point Method for Nonmonotone Variational Inequalities, Mathematical Methods of Operations Research 63 (2004), no. 3, 553–565. [2] T. Alpcan and T. Bas¸ar, A game-theoretic framework for congestion control in general topology networks, 41th IEEE Conf. Decision and Control (Las Vegas, NV), December 2002. [3] T. Alpcan and T. Basar, Distributed algorithms for nash equilibria of flow control games, Annals of Dynamic Games 7 (2003). [4] E. Altman, T. Basar, T. Jim´enez, and N. Shimkin, Competitive routing in networks with polynomial cost, INFOCOM (3), 2000, pp. 1586– 1593. [5] F. Facchinei and J-S. Pang, Finite dimensional variational inequalities and complementarity problems: Vols i and ii, Springer-Verlag, NY, Inc., 2003. [6] I. V. Konnov, Equilibrium models and variational inequalities, Elsevier, 2007. [7] Y. Pan and L. Pavel, Games with coupled propagated constraints in optical network with multi-link topologies, Automatica 45 (2009), 871–880. [8] L. Pavel, A noncooperative game approach to OSNR optimization in optical networks, IEEE Transactions on Automatic Control 51 (2006), no. 5, 848–852. [9] B.T. Polyak, Introduction to optimization, Optimization Software Inc., New York, 1987. [10] G. Scutari, F. Facchinei, J-S. Pang, and D.P. Palomar, Monotone games in communications: Theory, algorithms, and models, submitted to IEEE Transactions on Information Theory (April 2010). [11] H. Yin, U.V. Shanbhag, and P.G. Mehta, Nash equilibrium problems with congestion costs and shared constraints, Under second review at IEEE Transactions on Automatic Control.
