The Conjugate Gradient Method for Optimal Control Problems - MIT
132
IEEE TR.4KSACTIONS O N AUTOMATIC CONTROL, VOL.
AC-12,
NO.
2, APRIL 1967
The Conjugate Gradient Method for Optimal Control Problems
Absfract-This paper extends the conjugate gradient minimization method of Fletcher and Reeves to optimal control problems. The technique is directly applicable only to unconstrained problems; if terminal conditions and inequality constraints are present, theproblem must be converted to an unconstrained form; e.g., by penalty functions. Only the gradient trajectory, itsnorm, and one additional trajectory,theactualdirection of search,needbestored.These search directions are generated from past and present valuesof the objective and its gradient. Successive points are determinedby linear minimization down these directions, which are always directions of descent. Thus, the method tendsconverge, to even from poor approximations to theminimum. Since, near its m n im i um,a general nonlinear problem can be approximated byone with a linear system and quadratic objective, rate of convergence is studied by considering this case. Here, the directions of search are conjugate and hence the objective is minimized over an expanding sequence of sets. Also, the distance from the current point to the miminum is reduced at each step. Three examples are presented to compare the method with the method of steepest descent. Convergence of the proposed method is much more rapid in all cases. A comparison with a secondvariational technique is also given in Example 3.
Similar difficulties existed, until recent1)-, in the field of finitedimensionaloptimization, Le., mathematical programming. Hou-ever, in the past fen- years several rapid1)- convergentfinitedimensionalunconstrained minimization techniques have been developed. Among these are the method of Fletcher and Pori-ell [ 5 ] and the Fletcher-Reeves [6] adaptation of theconjugate gradient method of Hestenes and Stiefel [5]. Both these procedures generate conjugate directions of search and thereforeminimize a positive definite quadratic function of t z variables in n steps. In addition, the directions the generated are aln-ays directionsof descent and thus, for relativelysmoothfunctions,thefunctionvalue is decreased at each step. The combination of these properties implies that the methods converge rapidly to the nearest local minimum for a generalfunction of n variables.Experiencehas shown that both techniques converge much more rapidly, in general, than the method of steepest descent while requiring onll- function and gradient evaluations. I. INTRODL-CTIOS H I S P-APER presents an iterative procedure for JIoreover,theirstabilitypropertiesaresuperiorto those of second-order Selvtonmethods,andsecondsolving unconstrained optimal control problems. order derivatives are not required. Ofcourse, a generalformulation of the optimal Function space analogs of the steepest descent and control problem involves both terminal constraints on second-order Sewton techniques have beendeveloped thestatevariablesand inequalit!. constraintsonthe and applied to problems of optimal control. In particustateandcontrolvariables enforced alongtheentire lar, Kelly and Bryson [8], [ 9 ] , RIitter [lo], and others trajectory. Penalty functions have often been used to have developed steepest descent and second-order convert such problems to a sequence of “unconstrained” methods. However, the analogs of the conjugate direcproblems, i.e., problems with no terminal or intion techniques have not );et been forthcoming. Since equalityconstraints [1]-[4]. I t is evidentthatthe these methods are considered by some authors [SI, [ l l ] efficiencl- of these methods depends greatll- on the techto be the most powerful presently available forfinite nique used to solve the unconstrained optimal control dimensional minimization problems, it seems appropriproblem. Presently available techniques all have shortcomings. T h e convergence of steepest descent methods ate to consider their generalization to optimal control. This paperdescribesanextension of the Fletcheris often slon- [l ] n-hereas second-variational and Sen-Reeves [ 6 ] conjugate gradient method to function tonmethodsmaynotconverge at all. Thus there is strong motivation for developing more efficient means spaceproblems. The computational simplicity of this algorithm led to its selection. As in the steepest descent for solving unconstrained optimal control problems. method, the gradient trajectory must be computed and stored. Inaddition,theconjugategradienttechnique Manuscript received -1ugust 1 1 , 1966; revised December 30, 1966. requires the computation of the norm of the gradient T h i s work was supported in part bl- U. S. .Army Research Office Grant D4-AROD-31-124-6647 and by National Science Foundation andthestorage of oneothertrajectory,theactual Grant GK-600. direction of search. L. S. Lasdon is with the Division of Organizational Sciences, Operations Research Group and Systems Research Center, Case Institute Despite its simplicity, computational results illustrate of Technology, Cleveland, Ohio. of steepest deS. K. Mltter is with the Engineering Division and Systems Re- itsmarkedsuperiorit>-tothetnethod search Center, Case Institute of Technology, Ch-eland, Ohio. scent. These results are substantiated by theoretical X. D. Li,*arenis with the Dept. of ElectricalEngineering, Fern1 College of Engineering, Cleveland State Lniversity, Cleveland,Ohio. developments.
T
133
LASDON ET AL.: CONJUGATE GRADIENT METHOD
is straightforI t is shown here that the directions in function space T h e extension to the multicontrol case ward. generated by the conjugate gradient method are such t h a t the objective function is decreased a t each step. This leads to the results shon-n in Section I11 concerning B . -4 lgorithna convergence from arbitrary starting points as the numThe conjugate gradient algorithm requires the comber of iterations approaches infinity. The rate of con- putation of the gradient trajectory. Let vergence is bestinvestigatedbyconsideringtheperformance of the algorithm on a problem with a linear system and quadratic objective (Section 11:). Here the results of Xntosiewicz and Rheinboldt [12] appll;, show- where ing that the directionsof search are conjugate and hence the function is minimized over an expanding sequence of sets. In addition, the distance from the current estimate of the solution totheoptimalpointdecreases monotonically. Further results in the linear-quadratic case havebeen obtained.Forlinear-quadraticproblems in onestate Then the gradient is variable,theconjugategradientmethod achieves a value of the objective a t least as low as that obtained bythemethod of steepestdescent at eachiteration (assumingthesamestartingpoint for both). 4 1 ~ 0 , a Let u i ( t ) bethe i t h approximationtotheoptimal class of quadratic problems is exhibited for n-hich the control u o ( t ) . The corresponding gradient g(zli) is comconjugate gradient method finds the optimal solutionin puted by solvingthestateequations (2) and(3) fora finite number of steps. n-ards with ZL = u i t solving the adjoint equations ( 5 ) and Xone of the above properties is shared by the method of steepest descent, and this accounts for the rapid con- (6) backn-ards and then computing g ( u i ) from (7). The algorithm proceeds as follows: vergence of the conjugate gradient methodfor quadratic problems. This implies that for those general nonlinear z~~ = arbitrary (8) problems xvhich may be approximated by linear-quadgo = g ( 4 (9) raticproblemsneartheoptimum,theconvergence is so = - go. (10) rapid. Likemostotheriterativeprocedures, t h i s method Choose cannot distinguish between local and global minima. In a = ai to minimize J(ui asi) (11) general, the best that can be expected is eficient convergence to the bottomof whatever valley it starts in [6]. andthen T h e usual procedure for problems \vith local minima is Ui+l = 21; a;si (12) to rerun the method with different starting points. gi+l = g(21i+l) (13) 11. CONJUGATE GRADIEKT ALGORITHM Bi = ( g i + l , g i - J i ( g i , g i ) (14) 9 . Problem Formulation S ~ + I= - gi+l Bisi (13 Consider the following problem : xhere minimize J = +(x(tfj) (1) subject t o x = f(x, u, t ) (2) (g;, gi) = ''gi(t)gj(l)d[. (16) 10 x(t0) = c (3) r o t e t h a t t h e new direction of search si+l is not the where x is an rt vector, u is an v z . vector, and t o ,t f are negative gradient direction -gi+l, b u t is computed via fixed. I t is assumed that given a control u , (2) and ( 3 ) (15). The distance traveled in thisdirection is detercan be solved for a unique x = x ( u ) , and thus J = J ( u ) mined bythe one-dimensionalminimization in (11). is a function of u alone. Furthermore, the existence of Subsequentproofsassumethatthisminimization is the gradient af J ( u ) , V J ( u )= g ( u ) is assumed. The ob- carried outexactly.Inpractice,this is not possible. jective function (1) may include penalty function terms Sumerical experiencehas shon-n thatstepping down to account for constraints. the search direction until the objective startsincrease to For the remainder of the paper only the case of a in value and thenusing cubic interpolation gives reasonsingle control function u ( t ) (m= 1) n-ill be considered. able results.
+
+
+
J
134
CONTROL, APRIL 1967
AUTOMATIC TRANSACTIONS ON IEEE
I I I. COKVERGENCE
2) J ( u ) and g(u.)continuous are
Let the control ZL be an element of a Hilbert space H and J ( u ) a Frechetdifferentiablemappingfrom H to the real numbers. The conjugate gradient method when applied to J ( u ) generates directions s i which are always directions of descent, Le.,
3,
k
D2J(21* It)
exists and
I D2J(u7?z7 h ) I 5 tltll J z l I j ~ , h E Z3, and 4, (uli1 has a ‘luster point*“* sequence then the {up1 formed with arbitrary
1%
>0
the by conjugate gradient method has the following properties:
(17)
u 0
1) lim J(uI;) = J ( u * ) ; t-+
0)
and this assures that J ( z L )is decreased a t each step. These statements are proved below.
2)
lim g(uJ = g(u*) = 0. k+
0)
Theorem 1 If g ( u i ) = g i Z O then (Si,
g i d =0
and d
+
- J ( U ~ asi)
da
gi)
= (si,
=
- jigill?.
Proof: Let a = a i minimize J(ui+cusi). Then d
-J ( U i da
+ asi)
=
(Si,
gi+J = 0.
From (15)
( g i , si)
=
(gi, - g i
+ Bi-1si-d
+ p I-1 . (g,,.
= -
sI-1 . ) =
- \ l g . 1’I,’ 2 . I
Th.eorem 2
a* =
If gi#O then J(ui+d< J ( u i ) . Proof: Assume there exists no a > 0 such that J(ZCi
+ as;)
0 J(Ui
+ asO such that J(ui+asi) <J(ui). Since ai lim J(z&) 4 J ( z r 0 ) -lig(zdi)i12. ti-1 2m is chosen to minimize J(ui+asi), the theorem is proved. The sequence of real numbers [ J ( z L ~ is ) thus mono- Since J ( z L )is bounded below, tone decreasing and therefore has a limit J , in the exI ; 1 tended real numbers. lim - Trzi/g(ztt)iJ2 Also of interest is the limiting behavior of the set-.= j=l 2 quences ( u k }and (g,:}. Resultssimilartothosethat have been obtained for the method of steepest descent exists and is finite, hence [Is], [I41 are given below. lim jlg(zrk)\jz = 0.
(30)
0)
1
(31)
(32)
t- m
Theorem 3
If the following assumptions are made 1) J(zL)is bounded below
B>- assumption 4, { Z L ]~ contains a convergent subsequence { ,&}, with limit point zt*. Then continuity of g(u) implies t h a t g ( u * ) =O. From Theorem 2, J ( U ~ + ~ )
L.4SDOK ET .4L.: CONJUGATEGRADIENT
135
METHOD
< J(uk) and hence b>- the convergence of ZL!:and continuitp of J(u), property 1 follows. Computational experience has shown that methods n-hich decrease the function J a t each step \vi11 generally converge to the nearest local minimum. Since the function is generally convex in some neighborhood of a local minimum, this statement is supported by the following result.
Since the si are linearly independent, Bj-lCBj j = 1 , . . . , n and hence J ( u ) is minimized over an expanding sequence of sets. 3) The error vectorsy ; = u*-ui are decreased a t each step, i.e., - of the other existing finite dimensionalminimizationtechniquescanbesimilarly extended. For problems with inequalit!- constraints, an extension of the S'LTAIT procedure of Fiacco and ~ ~ " IIcCormick [19] appears promising. In particular, combining SI->IT n-ith conjugate gradients should yield a computationally useful algorithm. Such an extension is currently being investigated.
REFEREKCES
'O0I
STEEPEST DESCENT AFTER 20 ITERATIONS
40 20
n
2 -20
CONJUGATE GRADIENT AFTER 2 0 ITERATIONS
(3
Fig. 6.
Gradienttrajectories--esanlplethree.
min ~ ~ ( 1 0 ) .
(64)
Figure 5 comparestheconvergence of thesteepest descent,conjugategradients,andthesecondvariationalmethod used byIIerriam. -1s in the previous examples the conjugate gradient method is markedly superiortosteepestdescents.Thesecondvariational technique is faster than both but requires considerably more computation per iteration. Figure 6 compares the gradient trajectories for steepest descent and conjugate gradients.
[l] R. E. Koppand R. LicGill, "Sek-era1 trajectoryoptimization techniques,"in Compzding X e f l ~ o d sin Opti?,tizationProblenzs, A. 1.. Balakrishnanand L. \V. Seustadt,Eds. KewYork: .Academic, 1964, pp. 65-89. [2] H. J. Kelley, "1lethods of gradients,- in 0ptin1i;ationTechniques, G. Leitmann, Ed. Ne\\- York: -Academic, 1962, ch. 6. [3] 1i.. F. Denham and A. E. Bryson, "Optimal progratnming problems \\-ith inequalit!. constraints-11: Solution by steepest ascent," -41.43 J., vol. 2, pp. 25-34, January 1964. [4] R. lIcGill, "Optimal control, inequality state constraints, and the generalized Sewton-Raphson algorithm," J . SI.-lJI ox Control. ser. A , vol. 3 , no. 2 , pp. 291-?9S, 1965. [5] R. Fletcher and 11.J. D. Powell, "X rapid]!- con\-ergent descent methodfor nlinimization,' British Conrplcter J . , pp. 163-168, June 1963. [6] R. Fletcher and C. k1. Reeves. "Function minimization by conjugate gradients," British Computer J., pp. 149-154. July 1964. [7] 11.R. Hestenes and E. Stiefel, "1Iethods of conjugate gradients for solving linear s>-sterns,* J . Researrh. :\-BS, vol. 49, p. 109, 1952. [8] H. J. IielleJ-, "Gradient theor>- of optimal flight paths," -4m. Rocket SOL.J., ~-01.30, pp. 947-953, October 1960. [9] S. R.IIcRevnolds and A. E. Brvson, Tr., "*A successive sweep methodfor- solving optimal progranqming problems,' Pro;. J-4 CC, 1965. S. AIitter, "Successive approzitnation methods for the solution Azdomatica, vol. 3, pp. 133-149, of optimalcontrolproblems, 1966. XI. J. Box, "rl comparison of severalcurrentoptimization methods and the use of transformations in constrained problems," The Computer J . , vol. 9, pp. 67-78, LIay 1966. Szwcev of Sumerical -4nalgsis. S e w York: J.Todd,Ed., SIcGra\v-Hill, 1962. J. R. Rice, The dpprosiaration of Functions. Reading,1Iass.: Addison-\Yesley, 1964, vol. 1. H. B. Curry."Themethod of steepestdescent for nonlinear minimizationproblems," Qzrart. d p p l . Math., vol. 2 , pp. 258261, October 1941. \V. E. Langlois. "Conditions for termination of the method of steepest descent after a finite number of iterations," I B X J . Research and Der., pp. 98-99, January 1966. C. T. Leondes, Ed., -4dtnnces in ControlSxstems. \-en- York: .Academic,1964-1966, vol. 2. S. E. Dre!.fus, "Variationalproblemswithstatevariable inequalit!- constraints," R.ISD Rept. P-2605-1,.August1963. C. \I-. LIerriam, 111, "Direct computational methods for feeda n d Control. vol. 8 , pp. back control optimization," I~rjorn~atron 215-232,=Spril1965. .A. 1.. Fiaccoand G. P. lIcCormick."Thesequential unconstrained minilnization technique for nonlinear programming, a primal-dual method," Xuulrugement Srienre, vol. 10, pp. 360-366, J n n u a n - 1964.
IEEE TR.4KSACTIONS O N AUTOMATIC CONTROL, VOL.
AC-12,
NO.
2, APRIL 1967
The Conjugate Gradient Method for Optimal Control Problems
Absfract-This paper extends the conjugate gradient minimization method of Fletcher and Reeves to optimal control problems. The technique is directly applicable only to unconstrained problems; if terminal conditions and inequality constraints are present, theproblem must be converted to an unconstrained form; e.g., by penalty functions. Only the gradient trajectory, itsnorm, and one additional trajectory,theactualdirection of search,needbestored.These search directions are generated from past and present valuesof the objective and its gradient. Successive points are determinedby linear minimization down these directions, which are always directions of descent. Thus, the method tendsconverge, to even from poor approximations to theminimum. Since, near its m n im i um,a general nonlinear problem can be approximated byone with a linear system and quadratic objective, rate of convergence is studied by considering this case. Here, the directions of search are conjugate and hence the objective is minimized over an expanding sequence of sets. Also, the distance from the current point to the miminum is reduced at each step. Three examples are presented to compare the method with the method of steepest descent. Convergence of the proposed method is much more rapid in all cases. A comparison with a secondvariational technique is also given in Example 3.
Similar difficulties existed, until recent1)-, in the field of finitedimensionaloptimization, Le., mathematical programming. Hou-ever, in the past fen- years several rapid1)- convergentfinitedimensionalunconstrained minimization techniques have been developed. Among these are the method of Fletcher and Pori-ell [ 5 ] and the Fletcher-Reeves [6] adaptation of theconjugate gradient method of Hestenes and Stiefel [5]. Both these procedures generate conjugate directions of search and thereforeminimize a positive definite quadratic function of t z variables in n steps. In addition, the directions the generated are aln-ays directionsof descent and thus, for relativelysmoothfunctions,thefunctionvalue is decreased at each step. The combination of these properties implies that the methods converge rapidly to the nearest local minimum for a generalfunction of n variables.Experiencehas shown that both techniques converge much more rapidly, in general, than the method of steepest descent while requiring onll- function and gradient evaluations. I. INTRODL-CTIOS H I S P-APER presents an iterative procedure for JIoreover,theirstabilitypropertiesaresuperiorto those of second-order Selvtonmethods,andsecondsolving unconstrained optimal control problems. order derivatives are not required. Ofcourse, a generalformulation of the optimal Function space analogs of the steepest descent and control problem involves both terminal constraints on second-order Sewton techniques have beendeveloped thestatevariablesand inequalit!. constraintsonthe and applied to problems of optimal control. In particustateandcontrolvariables enforced alongtheentire lar, Kelly and Bryson [8], [ 9 ] , RIitter [lo], and others trajectory. Penalty functions have often been used to have developed steepest descent and second-order convert such problems to a sequence of “unconstrained” methods. However, the analogs of the conjugate direcproblems, i.e., problems with no terminal or intion techniques have not );et been forthcoming. Since equalityconstraints [1]-[4]. I t is evidentthatthe these methods are considered by some authors [SI, [ l l ] efficiencl- of these methods depends greatll- on the techto be the most powerful presently available forfinite nique used to solve the unconstrained optimal control dimensional minimization problems, it seems appropriproblem. Presently available techniques all have shortcomings. T h e convergence of steepest descent methods ate to consider their generalization to optimal control. This paperdescribesanextension of the Fletcheris often slon- [l ] n-hereas second-variational and Sen-Reeves [ 6 ] conjugate gradient method to function tonmethodsmaynotconverge at all. Thus there is strong motivation for developing more efficient means spaceproblems. The computational simplicity of this algorithm led to its selection. As in the steepest descent for solving unconstrained optimal control problems. method, the gradient trajectory must be computed and stored. Inaddition,theconjugategradienttechnique Manuscript received -1ugust 1 1 , 1966; revised December 30, 1966. requires the computation of the norm of the gradient T h i s work was supported in part bl- U. S. .Army Research Office Grant D4-AROD-31-124-6647 and by National Science Foundation andthestorage of oneothertrajectory,theactual Grant GK-600. direction of search. L. S. Lasdon is with the Division of Organizational Sciences, Operations Research Group and Systems Research Center, Case Institute Despite its simplicity, computational results illustrate of Technology, Cleveland, Ohio. of steepest deS. K. Mltter is with the Engineering Division and Systems Re- itsmarkedsuperiorit>-tothetnethod search Center, Case Institute of Technology, Ch-eland, Ohio. scent. These results are substantiated by theoretical X. D. Li,*arenis with the Dept. of ElectricalEngineering, Fern1 College of Engineering, Cleveland State Lniversity, Cleveland,Ohio. developments.
T
133
LASDON ET AL.: CONJUGATE GRADIENT METHOD
is straightforI t is shown here that the directions in function space T h e extension to the multicontrol case ward. generated by the conjugate gradient method are such t h a t the objective function is decreased a t each step. This leads to the results shon-n in Section I11 concerning B . -4 lgorithna convergence from arbitrary starting points as the numThe conjugate gradient algorithm requires the comber of iterations approaches infinity. The rate of con- putation of the gradient trajectory. Let vergence is bestinvestigatedbyconsideringtheperformance of the algorithm on a problem with a linear system and quadratic objective (Section 11:). Here the results of Xntosiewicz and Rheinboldt [12] appll;, show- where ing that the directionsof search are conjugate and hence the function is minimized over an expanding sequence of sets. In addition, the distance from the current estimate of the solution totheoptimalpointdecreases monotonically. Further results in the linear-quadratic case havebeen obtained.Forlinear-quadraticproblems in onestate Then the gradient is variable,theconjugategradientmethod achieves a value of the objective a t least as low as that obtained bythemethod of steepestdescent at eachiteration (assumingthesamestartingpoint for both). 4 1 ~ 0 , a Let u i ( t ) bethe i t h approximationtotheoptimal class of quadratic problems is exhibited for n-hich the control u o ( t ) . The corresponding gradient g(zli) is comconjugate gradient method finds the optimal solutionin puted by solvingthestateequations (2) and(3) fora finite number of steps. n-ards with ZL = u i t solving the adjoint equations ( 5 ) and Xone of the above properties is shared by the method of steepest descent, and this accounts for the rapid con- (6) backn-ards and then computing g ( u i ) from (7). The algorithm proceeds as follows: vergence of the conjugate gradient methodfor quadratic problems. This implies that for those general nonlinear z~~ = arbitrary (8) problems xvhich may be approximated by linear-quadgo = g ( 4 (9) raticproblemsneartheoptimum,theconvergence is so = - go. (10) rapid. Likemostotheriterativeprocedures, t h i s method Choose cannot distinguish between local and global minima. In a = ai to minimize J(ui asi) (11) general, the best that can be expected is eficient convergence to the bottomof whatever valley it starts in [6]. andthen T h e usual procedure for problems \vith local minima is Ui+l = 21; a;si (12) to rerun the method with different starting points. gi+l = g(21i+l) (13) 11. CONJUGATE GRADIEKT ALGORITHM Bi = ( g i + l , g i - J i ( g i , g i ) (14) 9 . Problem Formulation S ~ + I= - gi+l Bisi (13 Consider the following problem : xhere minimize J = +(x(tfj) (1) subject t o x = f(x, u, t ) (2) (g;, gi) = ''gi(t)gj(l)d[. (16) 10 x(t0) = c (3) r o t e t h a t t h e new direction of search si+l is not the where x is an rt vector, u is an v z . vector, and t o ,t f are negative gradient direction -gi+l, b u t is computed via fixed. I t is assumed that given a control u , (2) and ( 3 ) (15). The distance traveled in thisdirection is detercan be solved for a unique x = x ( u ) , and thus J = J ( u ) mined bythe one-dimensionalminimization in (11). is a function of u alone. Furthermore, the existence of Subsequentproofsassumethatthisminimization is the gradient af J ( u ) , V J ( u )= g ( u ) is assumed. The ob- carried outexactly.Inpractice,this is not possible. jective function (1) may include penalty function terms Sumerical experiencehas shon-n thatstepping down to account for constraints. the search direction until the objective startsincrease to For the remainder of the paper only the case of a in value and thenusing cubic interpolation gives reasonsingle control function u ( t ) (m= 1) n-ill be considered. able results.
+
+
+
J
134
CONTROL, APRIL 1967
AUTOMATIC TRANSACTIONS ON IEEE
I I I. COKVERGENCE
2) J ( u ) and g(u.)continuous are
Let the control ZL be an element of a Hilbert space H and J ( u ) a Frechetdifferentiablemappingfrom H to the real numbers. The conjugate gradient method when applied to J ( u ) generates directions s i which are always directions of descent, Le.,
3,
k
D2J(21* It)
exists and
I D2J(u7?z7 h ) I 5 tltll J z l I j ~ , h E Z3, and 4, (uli1 has a ‘luster point*“* sequence then the {up1 formed with arbitrary
1%
>0
the by conjugate gradient method has the following properties:
(17)
u 0
1) lim J(uI;) = J ( u * ) ; t-+
0)
and this assures that J ( z L )is decreased a t each step. These statements are proved below.
2)
lim g(uJ = g(u*) = 0. k+
0)
Theorem 1 If g ( u i ) = g i Z O then (Si,
g i d =0
and d
+
- J ( U ~ asi)
da
gi)
= (si,
=
- jigill?.
Proof: Let a = a i minimize J(ui+cusi). Then d
-J ( U i da
+ asi)
=
(Si,
gi+J = 0.
From (15)
( g i , si)
=
(gi, - g i
+ Bi-1si-d
+ p I-1 . (g,,.
= -
sI-1 . ) =
- \ l g . 1’I,’ 2 . I
Th.eorem 2
a* =
If gi#O then J(ui+d< J ( u i ) . Proof: Assume there exists no a > 0 such that J(ZCi
+ as;)
0 J(Ui
+ asO such that J(ui+asi) <J(ui). Since ai lim J(z&) 4 J ( z r 0 ) -lig(zdi)i12. ti-1 2m is chosen to minimize J(ui+asi), the theorem is proved. The sequence of real numbers [ J ( z L ~ is ) thus mono- Since J ( z L )is bounded below, tone decreasing and therefore has a limit J , in the exI ; 1 tended real numbers. lim - Trzi/g(ztt)iJ2 Also of interest is the limiting behavior of the set-.= j=l 2 quences ( u k }and (g,:}. Resultssimilartothosethat have been obtained for the method of steepest descent exists and is finite, hence [Is], [I41 are given below. lim jlg(zrk)\jz = 0.
(30)
0)
1
(31)
(32)
t- m
Theorem 3
If the following assumptions are made 1) J(zL)is bounded below
B>- assumption 4, { Z L ]~ contains a convergent subsequence { ,&}, with limit point zt*. Then continuity of g(u) implies t h a t g ( u * ) =O. From Theorem 2, J ( U ~ + ~ )
L.4SDOK ET .4L.: CONJUGATEGRADIENT
135
METHOD
< J(uk) and hence b>- the convergence of ZL!:and continuitp of J(u), property 1 follows. Computational experience has shown that methods n-hich decrease the function J a t each step \vi11 generally converge to the nearest local minimum. Since the function is generally convex in some neighborhood of a local minimum, this statement is supported by the following result.
Since the si are linearly independent, Bj-lCBj j = 1 , . . . , n and hence J ( u ) is minimized over an expanding sequence of sets. 3) The error vectorsy ; = u*-ui are decreased a t each step, i.e., - of the other existing finite dimensionalminimizationtechniquescanbesimilarly extended. For problems with inequalit!- constraints, an extension of the S'LTAIT procedure of Fiacco and ~ ~ " IIcCormick [19] appears promising. In particular, combining SI->IT n-ith conjugate gradients should yield a computationally useful algorithm. Such an extension is currently being investigated.
REFEREKCES
'O0I
STEEPEST DESCENT AFTER 20 ITERATIONS
40 20
n
2 -20
CONJUGATE GRADIENT AFTER 2 0 ITERATIONS
(3
Fig. 6.
Gradienttrajectories--esanlplethree.
min ~ ~ ( 1 0 ) .
(64)
Figure 5 comparestheconvergence of thesteepest descent,conjugategradients,andthesecondvariationalmethod used byIIerriam. -1s in the previous examples the conjugate gradient method is markedly superiortosteepestdescents.Thesecondvariational technique is faster than both but requires considerably more computation per iteration. Figure 6 compares the gradient trajectories for steepest descent and conjugate gradients.
[l] R. E. Koppand R. LicGill, "Sek-era1 trajectoryoptimization techniques,"in Compzding X e f l ~ o d sin Opti?,tizationProblenzs, A. 1.. Balakrishnanand L. \V. Seustadt,Eds. KewYork: .Academic, 1964, pp. 65-89. [2] H. J. Kelley, "1lethods of gradients,- in 0ptin1i;ationTechniques, G. Leitmann, Ed. Ne\\- York: -Academic, 1962, ch. 6. [3] 1i.. F. Denham and A. E. Bryson, "Optimal progratnming problems \\-ith inequalit!. constraints-11: Solution by steepest ascent," -41.43 J., vol. 2, pp. 25-34, January 1964. [4] R. lIcGill, "Optimal control, inequality state constraints, and the generalized Sewton-Raphson algorithm," J . SI.-lJI ox Control. ser. A , vol. 3 , no. 2 , pp. 291-?9S, 1965. [5] R. Fletcher and 11.J. D. Powell, "X rapid]!- con\-ergent descent methodfor nlinimization,' British Conrplcter J . , pp. 163-168, June 1963. [6] R. Fletcher and C. k1. Reeves. "Function minimization by conjugate gradients," British Computer J., pp. 149-154. July 1964. [7] 11.R. Hestenes and E. Stiefel, "1Iethods of conjugate gradients for solving linear s>-sterns,* J . Researrh. :\-BS, vol. 49, p. 109, 1952. [8] H. J. IielleJ-, "Gradient theor>- of optimal flight paths," -4m. Rocket SOL.J., ~-01.30, pp. 947-953, October 1960. [9] S. R.IIcRevnolds and A. E. Brvson, Tr., "*A successive sweep methodfor- solving optimal progranqming problems,' Pro;. J-4 CC, 1965. S. AIitter, "Successive approzitnation methods for the solution Azdomatica, vol. 3, pp. 133-149, of optimalcontrolproblems, 1966. XI. J. Box, "rl comparison of severalcurrentoptimization methods and the use of transformations in constrained problems," The Computer J . , vol. 9, pp. 67-78, LIay 1966. Szwcev of Sumerical -4nalgsis. S e w York: J.Todd,Ed., SIcGra\v-Hill, 1962. J. R. Rice, The dpprosiaration of Functions. Reading,1Iass.: Addison-\Yesley, 1964, vol. 1. H. B. Curry."Themethod of steepestdescent for nonlinear minimizationproblems," Qzrart. d p p l . Math., vol. 2 , pp. 258261, October 1941. \V. E. Langlois. "Conditions for termination of the method of steepest descent after a finite number of iterations," I B X J . Research and Der., pp. 98-99, January 1966. C. T. Leondes, Ed., -4dtnnces in ControlSxstems. \-en- York: .Academic,1964-1966, vol. 2. S. E. Dre!.fus, "Variationalproblemswithstatevariable inequalit!- constraints," R.ISD Rept. P-2605-1,.August1963. C. \I-. LIerriam, 111, "Direct computational methods for feeda n d Control. vol. 8 , pp. back control optimization," I~rjorn~atron 215-232,=Spril1965. .A. 1.. Fiaccoand G. P. lIcCormick."Thesequential unconstrained minilnization technique for nonlinear programming, a primal-dual method," Xuulrugement Srienre, vol. 10, pp. 360-366, J n n u a n - 1964.
