Extremum seeking control - Semantic Scholar
Extremum seeking control: convergence analysis
Dragan Nešić The University of Melbourne
Acknowledgements: Y. Tan, I. Mareels, A. Astolfi, G. Bastin, C. Manzie; Australian Research Council.
Outline z z z z z z
Motivating examples Problem formulation Background Non-local stability: No local extrema With local extrema Some open problems. Conclusions
Motivating example bioreactors
Continuously Stirred Tank (CST) Reactor
u=Vol. flow rate
Performance output y: Productivity JP Substrate
Product
Inflow
Outflow
Assumption: u(t) ´ u ¯ =)
Yield JY Overall JT JT := λJP + (1 ¡ λ)JY , λ 2 (0, 1)
J? (t) !
J? (¯ u)
Single enzymatic reaction Michaelis-Menten Kinetics
Productivity and yield
Total cost
In steady-state, we would typically want to operate around
JT (¯ u) is typically unknown!!
u∗
Other examples Plant
Performance output
Turbine
Generated power
Solar cell
Generated power
Optical amplifiers
Uniformity of the gain spectrum
Tokamak
Reflected power during Lower Hybrid (LH) plasma heating experiments
Non-holonomic vehicles
Distance from a source of a signal
Paper machine
Retention of fines and fibers in the sheet
Ultrasonic/Sonic Driller/Corer
Distance from resonance
Human Exercise Machine
The user’s power output
ABS
Magnitude of friction force
Variable cam timing
Fuel consumption
Problem formulation Assumption 1: - Q(.) has an extremum (max)
u
x˙
= f (x, u) y = Q(u) y = h(x)
y
y ∗ := Q(u∗ ) ¸ Q(u), 8u - Q(.) is unknown Dynamic case:
9`(¢) ) Extremum Seeking Controller
0
=
f (`(u), u)
Q(u)
:=
h ± `(u)
Problem: Design ESC so that
lim supt→∞ jy(t) ¡ y ∗ j ¼ 0
Background
Classification of approaches
Deterministic
Stochastic
Adaptive ESC
Adaptive ESC
[Krstić, Ariyur, Guay, Tan, Nešić,…]
NLP based ESC [Popović, Teel,…]
[Krstić, Manzie,…]
NLP based ESC [Spall,..]
Also continuous-time versus discrete-time.
1922
Beginning tE SC ?
1950 1960 1970
Ad-hoc designs
os e
d
Ås pr t r ö om m isi & W ng Fi ad itten rs t lo ap m ca tiv ark St ls e ab co : “on ta ilit bil nt e yp ro o ity lt ft re No r o o e c he su f hn m n fo i q os r a -loc for N lt fo ue t r da al L a P d pt s t a ap s”. sc ive b t ES ility hem ive ES es C an C al y sis
Vi M br a E s a ny n t re pe ne s e c ia w a r l ly s c h c h Ad em are ap e a tiv s p e ro p
Fi rs
Brief history (deterministic):
1995 2000 2009
Rigorous analysis and design
Adaptive ESC [Krstić & Wang 2000], local stability
u
x˙
=
f (x, u)
y
=
h(x)
y
θ Q(u) := h ± `(u)
+
Wl (s) a sin(ωt)
Extremum seeking controller
K s
x
Wh (s) a sin(ωt)
Our goals: Precise non-local convergence analysis.
Controller tuning guidelines and trade-offs.
Non-local stability (no local extrema)
Y. Tan, D. Nešić and I. Mareels, “On non-local stability properties of extremum seeking control”, Automatica, Vol. 42, No. 6, pp. 889-903, 2006.
Static SISO case (gradient descent)
u = θ + a sin(t)
Parameters:
y
y = Q(u)
a, δ
θ
+
θ a sin(t)
Extremum seeking controller
δ s
θ˙ δ
x
sin(t)
Notation:
Dk Q :=
dk Q duk
Average system z
The system is periodic in time: θ˙ = δQ(θ + a sin(t)) sin(t) =: δf (t, θ, a)
z
Its average is a gradient descent scheme: ˙θ = δfav (θ, a) = δ[ a DQ(θ) +O(a3 )] |2 {z } Gradient descent
fav (θ, a) :=
1 2π
Z
2π
f (τ, θ, a)
0
Assumption 2:
DQ(u)(u ¡ u∗ ) < 0, 8u 6 = u∗ y = Q(u) y∗ DQ(u) > 0
DQ(u) < 0
u u∗ This assumption holds for many plants, e.g. some models of CST reactor.
KL functions z
Linear UGES systems satisfy the bound jx(t)j · K exp(¡ λ(t ¡ t0 ))jx0 j, 8t ¸ t0 , 8x0
z
for some K, λ>0. Nonlinear UGAS systems satisfy jx(t)j · β(jx0 j, t ¡ t0 ), 8t ¸ t0 , 8x0
for some β ∈ KL.
Theorem: Suppose Assumptions 1 and 2 hold. Then, there exists β ∈ KL such that: 8(∆, ν)9(δ ∗ , a∗ ) + 8δ 2 (0, δ ∗ ), a 2 (0, a∗ )
Tuning guidelines
+ jθ(t0 ) ¡ θ∗ j · ∆ + jθ(t) ¡ θ∗ j · β(jθ(t0 ) ¡ θ∗ j, aδ(t ¡ t0 )) + ν, 8t ¸ t0
where
θ∗ := u∗ .
We say that the system in SPA stable in a, δ.
A trade-off
Larger ∆ or ) Smaller ν
Smaller a ) and Smaller δ
Slower Convergence
Sketch of proof: z
Use the Lyapunov function candidate V (θ) = DV (θ)δfav (θ, a) =
1 (θ ¡ θ∗ )2 2⎡
⎤
a ⎣ δ DQ(θ)(θ ¡ θ ∗ ) +O(a3 )⎦ {z } 2| Q(u), 8u 6 = u∗ .
Static SISO case u = θ + a(t) sin(t)
y
y = Q(u)
Parameters:
a0 , δ, ²
θ
+
θ
δ s
θ˙ δ
x
sin(t)
x
a˙ = ¡ ²δa, a(0) = a0 > 0 Extremum seeking controller
Model of the system z
The system is time-varying: θ˙
=
δQ(θ + a sin(t)) sin(t) =: δf (t, θ, a)
a˙
=
¡ ²δa, a(0) = a0
and its average with a change of time σ=t/² dθ ² dσ da dσ
=
δfav (θ, a)
=
¡ δa, a(0) = a0
is a singularly perturbed system.
Desired bifurcation diagram a a = μ(θ)
fav (θ, a) = 0, a > 0 Global maximum θ ∗ ∗ ∗ Local maxima θ1 , θ2 , . . . Local minima ψ1∗ , ψ2∗ , . . .
a∗
θ θ1∗
ψ1∗
θ∗
ψ2∗
θ2∗
Assumption 5: The average system fav(θ,a) has a desired bifurcation diagram.
Comments z
All 4th order polynomials that satisfy Assumption 4 also satisfy Assumption 5.
z
There exists a 6th order polynomial that satisfies Assumption 4 but does not satisfy Assumption 5.
z
Dither shape affects Assumption 5!
Theorem z
Suppose Assumptions 4 and 5 hold. Then 8(∆, ν), a0 > a∗ +
9²∗ > 0, 8² 2 (0, ²∗ ) + 9δ ∗ > 0, 8δ 2 (0, δ ∗ )
Tuning guidelines
+ jθ0 ¡ θ∗ j · ∆
+ jθ(t) ¡ μ(a(t))j
·
β(jθ0 ¡ μ(a0 )j, δ(t ¡ t0 )) + ν
ja(t)j
·
exp(¡ ²δ(t ¡ t0 ))ja0 j
Comments z
Note that a(t) !
z
0 ) limt→∞ μ(a(t)) = μ(0) = θ ∗
To achieve robustness, we would typically modify ESC so that limt→∞ a(t) = a ¯>0
z
Similar to “simulated annealing”.
Idea a
Pick any a0 > a∗ Pick sufficiently small
a = μ(θ)
a0 Fast transient Desired Accuracy given
(θ0 , a0 )
Slow transient
a∗
θ∗ Desired domain of attraction given
θ0
θ
², δ
Comments z
Assumptions are impossible to verify a priori.
z
Our result provides a tuning strategy for ESC that can improve performance.
Some open problems z z z z
z z
Convergence rate improvements. Using the model knowledge in the best way. Adaptive versions of non-gradient schemes. Selection of efficient algorithms and dithers for particular applications. More detailed tuning guidelines, and so on. Multi-valued functions.
Multi-valued functions
G. Bastin, D. Nešić, Y. Tan and I. Mareels, “On Extremum Seeking in Bioprocesses with Multi-valued Cost Functions”, Biotechnology Progress, Vol. 25, No. 3, pp. 683-689, 2009.
Multi-valued cost z
Our assumptions sometimes do not hold.
JP is a multi-valued function
For some initial conditions our analysis is fine
Possible situations
Effects of “small” amplitude
Amplitude “too large”
Conclusions z
z z z z
Non-local convergence analysis of a class of adaptive ES controllers is presented. Tuning guidelines follow from our results. Interesting trade-offs arise. Global ES possible with local extrema. Many open problems.
Dragan Nešić The University of Melbourne
Acknowledgements: Y. Tan, I. Mareels, A. Astolfi, G. Bastin, C. Manzie; Australian Research Council.
Outline z z z z z z
Motivating examples Problem formulation Background Non-local stability: No local extrema With local extrema Some open problems. Conclusions
Motivating example bioreactors
Continuously Stirred Tank (CST) Reactor
u=Vol. flow rate
Performance output y: Productivity JP Substrate
Product
Inflow
Outflow
Assumption: u(t) ´ u ¯ =)
Yield JY Overall JT JT := λJP + (1 ¡ λ)JY , λ 2 (0, 1)
J? (t) !
J? (¯ u)
Single enzymatic reaction Michaelis-Menten Kinetics
Productivity and yield
Total cost
In steady-state, we would typically want to operate around
JT (¯ u) is typically unknown!!
u∗
Other examples Plant
Performance output
Turbine
Generated power
Solar cell
Generated power
Optical amplifiers
Uniformity of the gain spectrum
Tokamak
Reflected power during Lower Hybrid (LH) plasma heating experiments
Non-holonomic vehicles
Distance from a source of a signal
Paper machine
Retention of fines and fibers in the sheet
Ultrasonic/Sonic Driller/Corer
Distance from resonance
Human Exercise Machine
The user’s power output
ABS
Magnitude of friction force
Variable cam timing
Fuel consumption
Problem formulation Assumption 1: - Q(.) has an extremum (max)
u
x˙
= f (x, u) y = Q(u) y = h(x)
y
y ∗ := Q(u∗ ) ¸ Q(u), 8u - Q(.) is unknown Dynamic case:
9`(¢) ) Extremum Seeking Controller
0
=
f (`(u), u)
Q(u)
:=
h ± `(u)
Problem: Design ESC so that
lim supt→∞ jy(t) ¡ y ∗ j ¼ 0
Background
Classification of approaches
Deterministic
Stochastic
Adaptive ESC
Adaptive ESC
[Krstić, Ariyur, Guay, Tan, Nešić,…]
NLP based ESC [Popović, Teel,…]
[Krstić, Manzie,…]
NLP based ESC [Spall,..]
Also continuous-time versus discrete-time.
1922
Beginning tE SC ?
1950 1960 1970
Ad-hoc designs
os e
d
Ås pr t r ö om m isi & W ng Fi ad itten rs t lo ap m ca tiv ark St ls e ab co : “on ta ilit bil nt e yp ro o ity lt ft re No r o o e c he su f hn m n fo i q os r a -loc for N lt fo ue t r da al L a P d pt s t a ap s”. sc ive b t ES ility hem ive ES es C an C al y sis
Vi M br a E s a ny n t re pe ne s e c ia w a r l ly s c h c h Ad em are ap e a tiv s p e ro p
Fi rs
Brief history (deterministic):
1995 2000 2009
Rigorous analysis and design
Adaptive ESC [Krstić & Wang 2000], local stability
u
x˙
=
f (x, u)
y
=
h(x)
y
θ Q(u) := h ± `(u)
+
Wl (s) a sin(ωt)
Extremum seeking controller
K s
x
Wh (s) a sin(ωt)
Our goals: Precise non-local convergence analysis.
Controller tuning guidelines and trade-offs.
Non-local stability (no local extrema)
Y. Tan, D. Nešić and I. Mareels, “On non-local stability properties of extremum seeking control”, Automatica, Vol. 42, No. 6, pp. 889-903, 2006.
Static SISO case (gradient descent)
u = θ + a sin(t)
Parameters:
y
y = Q(u)
a, δ
θ
+
θ a sin(t)
Extremum seeking controller
δ s
θ˙ δ
x
sin(t)
Notation:
Dk Q :=
dk Q duk
Average system z
The system is periodic in time: θ˙ = δQ(θ + a sin(t)) sin(t) =: δf (t, θ, a)
z
Its average is a gradient descent scheme: ˙θ = δfav (θ, a) = δ[ a DQ(θ) +O(a3 )] |2 {z } Gradient descent
fav (θ, a) :=
1 2π
Z
2π
f (τ, θ, a)
0
Assumption 2:
DQ(u)(u ¡ u∗ ) < 0, 8u 6 = u∗ y = Q(u) y∗ DQ(u) > 0
DQ(u) < 0
u u∗ This assumption holds for many plants, e.g. some models of CST reactor.
KL functions z
Linear UGES systems satisfy the bound jx(t)j · K exp(¡ λ(t ¡ t0 ))jx0 j, 8t ¸ t0 , 8x0
z
for some K, λ>0. Nonlinear UGAS systems satisfy jx(t)j · β(jx0 j, t ¡ t0 ), 8t ¸ t0 , 8x0
for some β ∈ KL.
Theorem: Suppose Assumptions 1 and 2 hold. Then, there exists β ∈ KL such that: 8(∆, ν)9(δ ∗ , a∗ ) + 8δ 2 (0, δ ∗ ), a 2 (0, a∗ )
Tuning guidelines
+ jθ(t0 ) ¡ θ∗ j · ∆ + jθ(t) ¡ θ∗ j · β(jθ(t0 ) ¡ θ∗ j, aδ(t ¡ t0 )) + ν, 8t ¸ t0
where
θ∗ := u∗ .
We say that the system in SPA stable in a, δ.
A trade-off
Larger ∆ or ) Smaller ν
Smaller a ) and Smaller δ
Slower Convergence
Sketch of proof: z
Use the Lyapunov function candidate V (θ) = DV (θ)δfav (θ, a) =
1 (θ ¡ θ∗ )2 2⎡
⎤
a ⎣ δ DQ(θ)(θ ¡ θ ∗ ) +O(a3 )⎦ {z } 2| Q(u), 8u 6 = u∗ .
Static SISO case u = θ + a(t) sin(t)
y
y = Q(u)
Parameters:
a0 , δ, ²
θ
+
θ
δ s
θ˙ δ
x
sin(t)
x
a˙ = ¡ ²δa, a(0) = a0 > 0 Extremum seeking controller
Model of the system z
The system is time-varying: θ˙
=
δQ(θ + a sin(t)) sin(t) =: δf (t, θ, a)
a˙
=
¡ ²δa, a(0) = a0
and its average with a change of time σ=t/² dθ ² dσ da dσ
=
δfav (θ, a)
=
¡ δa, a(0) = a0
is a singularly perturbed system.
Desired bifurcation diagram a a = μ(θ)
fav (θ, a) = 0, a > 0 Global maximum θ ∗ ∗ ∗ Local maxima θ1 , θ2 , . . . Local minima ψ1∗ , ψ2∗ , . . .
a∗
θ θ1∗
ψ1∗
θ∗
ψ2∗
θ2∗
Assumption 5: The average system fav(θ,a) has a desired bifurcation diagram.
Comments z
All 4th order polynomials that satisfy Assumption 4 also satisfy Assumption 5.
z
There exists a 6th order polynomial that satisfies Assumption 4 but does not satisfy Assumption 5.
z
Dither shape affects Assumption 5!
Theorem z
Suppose Assumptions 4 and 5 hold. Then 8(∆, ν), a0 > a∗ +
9²∗ > 0, 8² 2 (0, ²∗ ) + 9δ ∗ > 0, 8δ 2 (0, δ ∗ )
Tuning guidelines
+ jθ0 ¡ θ∗ j · ∆
+ jθ(t) ¡ μ(a(t))j
·
β(jθ0 ¡ μ(a0 )j, δ(t ¡ t0 )) + ν
ja(t)j
·
exp(¡ ²δ(t ¡ t0 ))ja0 j
Comments z
Note that a(t) !
z
0 ) limt→∞ μ(a(t)) = μ(0) = θ ∗
To achieve robustness, we would typically modify ESC so that limt→∞ a(t) = a ¯>0
z
Similar to “simulated annealing”.
Idea a
Pick any a0 > a∗ Pick sufficiently small
a = μ(θ)
a0 Fast transient Desired Accuracy given
(θ0 , a0 )
Slow transient
a∗
θ∗ Desired domain of attraction given
θ0
θ
², δ
Comments z
Assumptions are impossible to verify a priori.
z
Our result provides a tuning strategy for ESC that can improve performance.
Some open problems z z z z
z z
Convergence rate improvements. Using the model knowledge in the best way. Adaptive versions of non-gradient schemes. Selection of efficient algorithms and dithers for particular applications. More detailed tuning guidelines, and so on. Multi-valued functions.
Multi-valued functions
G. Bastin, D. Nešić, Y. Tan and I. Mareels, “On Extremum Seeking in Bioprocesses with Multi-valued Cost Functions”, Biotechnology Progress, Vol. 25, No. 3, pp. 683-689, 2009.
Multi-valued cost z
Our assumptions sometimes do not hold.
JP is a multi-valued function
For some initial conditions our analysis is fine
Possible situations
Effects of “small” amplitude
Amplitude “too large”
Conclusions z
z z z z
Non-local convergence analysis of a class of adaptive ES controllers is presented. Tuning guidelines follow from our results. Interesting trade-offs arise. Global ES possible with local extrema. Many open problems.
