Adaptive Methods for Sequential Importance Sampling with ...
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Adaptive Methods for Sequential Importance Sampling with Application to State Space Models
2
J. Cornebise1, 2 , É. Moulines1 , J. Olsson3 1 Télécom ParisTech, France University Pierre and Marie Curie - Paris 6, France 3 Lund University, Sweden
IWAP 2008 July 8th , 2008 Compiègne, France
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Outline
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Particle filters: Sequentially iterate two steps
Particle filters, a.k.a. Sequential Monte Carlo (SMC): approximate a sequence of probability distributions by a sequence of discrete probability distributions. Support points of the discrete distributions are called particles, weighted by their mass. Sequentially iterate mutation of the particles and selection. Seminal paper by Gordon et al. (1993). Deep theoretical study: see Chopin (2002), Del Moral (2004), Douc and Moulines (2007), and practical approach, Doucet et al. (2001). Wide litterature to improve the basic algorithms. Variants in the way the two key steps are gone through, how often resampling is used. Tuning parameters: number of particles, resampling algorithm, number of offsprings for each particle in proposal step, proposition kernel.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
N We have: weighted sample {(ξN,i , ωN,i )}M i=1 targeting ν ∈ P(Ξ). More on targeting
We want: new weighted sample approximating R L(ξ, ·) ν(dξ) νL(·) µ(·) , = R Ξ ˜ ˜ ν(dξ 0 ) νL(Ξ) L(ξ 0 , Ξ) Ξ ˜ B(Ξ)), ˜ on some other state space (Ξ, with L finite transition kernel from ˜ ˜ (Ξ, B(Ξ)) to (Ξ, B(Ξ)). Natural strategy: approximate µ by: MN
µN (·) ,
X i=1
h i ˜ ωN,i L(ξN,i , Ξ) ˜ . L(ξN,i , ·)/L(ξN,i , Ξ) PMN ˜ j=1 ωN,j L(ξN,j , Ξ)
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Auxiliary Sampling Importance Sampling Illustration Adaptation
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
State space models Typical field of applications: state space models, Hidden Markov Models. ·|X
Xk+1 = fk (Xk , Wk+1 ) ∼k Kk (Xk , ·), ·|Xk
Yk = hk (Xk , Vk ) ∼ gk (·, Xk )λ(·),
k≥0, k≥0,
where ∞ {fk }∞ k=0 , {hk }k=0 sets of known R-valued functions ∞ {Wk }∞ k=1 , {Vk }k=0 are mutually independent sets of N (0, 1) s.t. Wk+1 ⊥ ⊥ {(Xi , Yi )}ki=0 and Vk ⊥ ⊥ Xk and Vk ⊥ ⊥ {(Xi , Yi )}k−1 i=0 . ∞ {Kk }∞ k=0 family of Markov kernels, prior kernels, and {gk }k=0 densities w.r.t. a measure λ, local likelihood.
Aim: approximating filtering measure φk (u) , E [u(Xk )|Y0:k = y0:k ], for possibly several B(Xt+1 )/B(R)-measurable functions of interest u. ˜ Solution: SMC k , µ = φk+1 , and R with Ξ0 = Ξ = R, ν = φ Lk (ξ, A) = A gk+1 (ξ , Yk+1 ) Kk (ξ, dξ 0 ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Direct simulation from µN most often expensive (Accept/Reject, Hurzeler and Kunsch (1998), Kunsch (2005)). ˜N Solution: simulate new particles {ξ˜N,i }M i=1 from instrumental mixture MN
πN (·) ,
X i=1
ωN,i ψN,i R(ξN,i , ·) , PMN j=1 ωN,j ψN,j
where N {ψN,i = Ψ(ξN,i )}M i=1 positive numbers, adjustment multiplier weights,
R is a Markovian kernel, proposal kernel, ˜ importance weights {dµN /dπN (ξ˜N,i )}MN . i=1
Still, dµN /dπN expensive to evaluate: summing over MN terms.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j 2
Assign each draw (IN,i , ξ˜N,i ) the weight ω ˜ N,i , Φ(ξN,IN,i , ξ˜N,i ), where ˜ , Ψ(ξ)−1 Φ(ξ, ξ) second stage weights, proportional to
J. Cornebise, É. Moulines, J. Olsson
dL(ξ, ·) ˜ (ξ) , dR(ξ, ·)
dµaux N (IN,i , ξ˜N,i ). aux dπN
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j 2
Assign each draw (IN,i , ξ˜N,i ) the weight ω ˜ N,i , Φ(ξN,IN,i , ξ˜N,i ), where ˜ , Ψ(ξ)−1 Φ(ξ, ξ) second stage weights, proportional to
3
dL(ξ, ·) ˜ (ξ) , dR(ξ, ·)
dµaux N (IN,i , ξ˜N,i ). aux dπN ˜
N Marginalize: discard the indices and take {(ξ˜N,i , ω ˜ N,i )}M i=1 as an approximation of µ.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Skip Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
To beginning Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Adaptation: past and present
Aim: Ease the role of the user by automatically tuning the key parameters of the SMC algorithm. Past work: Important topic, previous approaches adapting: Number of particles: Legland and Oudjane (2006), and later Hu, Schon and Ljung (to appear), avoid degeneracy of the weights; also KLD-Sampling by Fox (2003), refined in Soto (2005) and Straka and Simandl (2006). Proposal kernel: Pitt and Shephard (1999) and Doucet, Godsill and Andrieu (2000), approximate so-called optimal kernel; Chan, Doucet and Tadic (2003) minimize expectation of a cost function, e.g. MSE or effective sample size over a parametric family of kernels. All rely on common criterions to assess the quality of the swarm. Sequential setting: function-free criterions required – due to recursivity of variance.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Definitions and criterions Asymptotics of criterions Optimal weights
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Common measures of discrepancy
Two probability measures µ and ν in P(Λ), s.t. µ ν. Recall the Kullback-Leibler Discrepancy (KLD) and Chi-Square Discrepancy (CSD): Definition: KLD and CSD Z dKL (µ||ν) ,
log ZΛ
dχ2 (µ||ν) ,
[ Λ
J. Cornebise, É. Moulines, J. Olsson
dµ (λ) µ(dλ) , dν
dµ (λ) − 1]2 ν(dλ) . dν
Adaptive refueling in SMC
Definitions and criterions Asymptotics of criterions Optimal weights
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Quality criterions aux Empirical estimates of the KLD and CSD between µaux and πN . N
Quality criterions
˜
˜N M
MN ˜ −1 E({˜ ωN,i }i=1 ),Ω N
X
“ ” ˜NΩ ˜ −1 ω ˜ N,i log M ˜ N,i , N ω
i=1
CV
2
˜N ({˜ ωN,i }M i=1 )
˜N M
˜NΩ ˜ −2 ,M N
X
2 ω ˜ N,i −1.
i=1
˜ N = PM˜ N ω where Ω i=1 ˜ N,i . ˜ N ). E: negated Shannon entropy of the importance weights (note the M CV2 : square of the widely used coefficient of variation of the importance weights, suggested by Kong, Liu and Wong (1994) and Liu and Chen (1995). Both minimal when all the weights are equal, maximal when all of the weights are zero except one. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
Two fundamental measures ˜ Define, for any A ∈ (B(Ξ) ⊗ B(Ξ)): ν⊗L (A) ˜ νL(Ξ) ν[Ψ] ⊗ R ∗ πΨ (A) , (A) ν(Ψ) µ∗ (A) ,
where the outer product of a measure and a kernel is defined as ZZ ν ⊗ L(A) , ν(dξ) L(ξ, dξ 0 )1A (ξ, ξ 0 ) , ˜ Ξ×Ξ
and a measure weighted by a function Ψ ∈ B(Ξ) is defined as ν[Ψ](B) , ν(Ψ1B ) , for B ∈ B(Ξ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We have the following limits, as N → ∞. Theorem Under weak technical conditions on the kernels and the weight function, 1
Asymptotic behavior of KLD: aux ∗ ∗ dKL (µaux N ||πN ) −→ dKL ( µ k πΨ ) , P
2
Asymptotic behavior of CSD: aux ∗ ∗ dχ2 (µaux N ||πN ) −→ dχ2 ( µ k πΨ ) . P
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We have the same limits for E and CV2 , as N → ∞. Theorem Under the same conditions, 1
Asymptotic behavior of KLD-based criterion: ˜
∗ ∗ N E({˜ ωN,i }M i=1 ) −→ dKL ( µ k πΨ ) , 2
P
Asymptotic behavior of CSD-based criterion: ˜
∗ ∗ N CV2 ({˜ ωN,i }M i=1 ) −→ dχ2 ( µ k πΨ ) .
J. Cornebise, É. Moulines, J. Olsson
P
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
Explicit forms for optimal weights
From the previous limiting expressions, we exhibit explicit forms for optimal adjustment multiplier weights. Corollary Under weak technical conditions, 1
The optimal weight function for KLD-based adaptation is aux ∗ ˜ arg min dKL (µaux N ||πN ) = ΨKL,R (ξ) , L(ξ, Ξ) . Ψ
2
The optimal weight function for CSD-based adaptation is sZ dL(ξ, ·) ˜ aux aux ∗ ˜ . (ξ) L(ξ, dξ) arg min dχ2 (µN ||πN ) = Ψχ2 ,R (ξ) , ˜ dR(ξ, ·) Ψ Ξ
KLD optimal weights independant of R. Special case: R ∝ L.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
aux These insights lead to adaptive design of πN ,
Key idea Minimize the criterions over some parametric family {Rθ }θ∈Θ of proposal kernels. Leads to auxiliary proposal ωN,i ψN,i aux πN,θ ({i} × A) , PM Rθ (ξN,i , A) . N j=1 ωN,j ψN,j
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
aux These insights lead to adaptive design of πN ,
Key idea Minimize the criterions over some parametric family {Rθ }θ∈Θ of proposal kernels. Leads to auxiliary proposal ωN,i ψN,i aux πN,θ ({i} × A) , PM Rθ (ξN,i , A) . N j=1 ωN,j ψN,j Two approaches: 1
Minimize E and CV2 , fixed seeds (non detailed here).
2
Minimize MN aux dKL (µaux N ||πN,θ )
=
XZ i=1
log ˜ Ξ
! dµaux N ˜ ˜ (i, ξ) µaux N (i, dξ) . aux dπN,θ
via a Cross-Entropy approach: iterative Importance Sampling approximation. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Cross Entropy-based adaptation ` Iteration `, current fit θN ∈ Θ. 1
` aux ` ` )} from πN,θ IS approximation (E): Sample MN particles {(IN,i , ξ˜N,i ` .
2
Optimization (M): Compute exact solution to the approximation
N
˜` M N
`+1 θN
` Xω ˜ N,i log , arg min ˜ Ω` θ∈Θ i=1
N
! dµaux ` ` N (IN,i , ξ˜N,i ) , aux dπN,θ
or equivalently, in presence of densities, ˜` M N `+1 θN , arg max θ∈Θ
` Xω ˜ N,i ` log rθ (ξN,I ` , ξ˜N,i ). N,i ˜` Ω i=1
N
where rθ (ξ, ·) density of Rθ (ξ, ·). Choose family {Rθ } s.t. closed-form update: Normal Exponential Family. Theoretical justication in progress, close resemblance with Monte-Carlo EM algorithms. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Benckmark: comparative study of the Mean-Square Error (MSE) on first order time-homogeneous autoregressions observed in noise: Reminder Xk+1 = m(Xk ) + σw (Xk )Wk+1 ,
k≥0,
Yk = Xk + σv Vk ,
For these very simple models, explicit forms for: ˜ – also optimal KLD weights for any Optimal weight Ψ∗ (ξ) , L(ξ, Ξ) kernel Ψ∗KL,R (ξ) ˜ Optimal kernel R∗ (ξ, ·) , L(ξ, ·)/L(ξ, Ξ) Optimal CSD weights for prior kernel Ψ∗χ2 ,K (ξ)
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Explicit forms - for sake of curiosity Optimal weight Ψ∗ (ξ) Ψ∗ (ξ) = N (Yk+1 ; m(ξ),
p 2 (ξ) + σ 2 ) σw v
Optimal kernel R∗ (ξ, ·) has density r∗ (ξ, ξ 0 ) = N (x0 ; τ (ξ, Yk+1 ), η(ξ)) , with τ (ξ, Yk+1 ) ,
2 σw (ξ)Yk+1 + σv2 m(ξ) , 2 (ξ) + σ 2 σw v
η 2 (ξ) ,
2 σw (ξ)σv2 2 σw (ξ) + σv2
.
Important: known mode ! Optimal CSD weights for prior kernel Ψ∗χ2 ,K (ξ) s Ψ∗χ2 ,K (ξ)
∝
„ « 2 Yk+1 m(ξ) 2σv2 ×exp − 2 + 2 [2Yk+1 − m(ξ)] . 2 (ξ) + σ 2 2σw σv 2σw (ξ) + σv2 v
We recall that Ψ∗KL,K (ξ) = Ψ∗k (ξ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
CE-update formulae Intentionally simple example, adaptation over n o Rθ (ξ, ·) , N (τ (ξ, Yk+1 ), θ η(ξ)) : ξ ∈ R, θ > 0 . ∗ aux Hence known optimal value θN , arg minθ>0 dKL (µaux N ||πN,θ ) = 1. Pretend we ignore it. Available KLD: MN aux dKL (µaux N ||πN,θ ) =
X i=1
" × log
∗ ωN,i ψN,i PMN ∗ j=1 ωN,j ψN,j
!
∗ ψN,i ΩN
PMN
j=1
∗ ωN,j ψN,j
Closed-form update formula for CE: ( M` ` θN N X ω ˜ N,i `+1 θN = ` ˜ θN η 2 ` i=1 Ω N θ
` θN ξ˜N,i
1 + log θ + 2
«# 1 −1 , θ2
!2 )1/2 −τ
θ`
N IN,i
N IN,i
2 where τN,i , τ (ξN,i , Yk+1 ) and ηN,i , η 2 (ξN,i ). J. Cornebise, É. Moulines, J. Olsson
„
Adaptive refueling in SMC
,
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
ARCH model Gaussian Auto-Regressive Conditional Heteroscedasticity model observed in p noise. mk (ξ) ≡ 0, σw (ξ) = β0 + β1 ξ 2 , i.e. q Xk+1 = β0 + β1 Xk2 Wk+1 , k ≥ 0 , Yk = Xk + σv Vk ,
k≥0.
1
Plain nonadaptive bootstrap particle filter (Ψ ≡ 1),
2
Optimal filter, i.e. optimal weights Ψ∗ and optimal kernel R∗ ,
3
Auxiliary filter with optimal χ2 weights Ψ∗χ2 ,K , prior kernel,
4
∗ Bootstrap filter with θN (known to be 1),
5
CE-based bootstrap filter (at last !).
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Adaptive Methods for Sequential Importance Sampling with Application to State Space Models
2
J. Cornebise1, 2 , É. Moulines1 , J. Olsson3 1 Télécom ParisTech, France University Pierre and Marie Curie - Paris 6, France 3 Lund University, Sweden
IWAP 2008 July 8th , 2008 Compiègne, France
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Outline
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Particle filters: Sequentially iterate two steps
Particle filters, a.k.a. Sequential Monte Carlo (SMC): approximate a sequence of probability distributions by a sequence of discrete probability distributions. Support points of the discrete distributions are called particles, weighted by their mass. Sequentially iterate mutation of the particles and selection. Seminal paper by Gordon et al. (1993). Deep theoretical study: see Chopin (2002), Del Moral (2004), Douc and Moulines (2007), and practical approach, Doucet et al. (2001). Wide litterature to improve the basic algorithms. Variants in the way the two key steps are gone through, how often resampling is used. Tuning parameters: number of particles, resampling algorithm, number of offsprings for each particle in proposal step, proposition kernel.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
N We have: weighted sample {(ξN,i , ωN,i )}M i=1 targeting ν ∈ P(Ξ). More on targeting
We want: new weighted sample approximating R L(ξ, ·) ν(dξ) νL(·) µ(·) , = R Ξ ˜ ˜ ν(dξ 0 ) νL(Ξ) L(ξ 0 , Ξ) Ξ ˜ B(Ξ)), ˜ on some other state space (Ξ, with L finite transition kernel from ˜ ˜ (Ξ, B(Ξ)) to (Ξ, B(Ξ)). Natural strategy: approximate µ by: MN
µN (·) ,
X i=1
h i ˜ ωN,i L(ξN,i , Ξ) ˜ . L(ξN,i , ·)/L(ξN,i , Ξ) PMN ˜ j=1 ωN,j L(ξN,j , Ξ)
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Auxiliary Sampling Importance Sampling Illustration Adaptation
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
State space models Typical field of applications: state space models, Hidden Markov Models. ·|X
Xk+1 = fk (Xk , Wk+1 ) ∼k Kk (Xk , ·), ·|Xk
Yk = hk (Xk , Vk ) ∼ gk (·, Xk )λ(·),
k≥0, k≥0,
where ∞ {fk }∞ k=0 , {hk }k=0 sets of known R-valued functions ∞ {Wk }∞ k=1 , {Vk }k=0 are mutually independent sets of N (0, 1) s.t. Wk+1 ⊥ ⊥ {(Xi , Yi )}ki=0 and Vk ⊥ ⊥ Xk and Vk ⊥ ⊥ {(Xi , Yi )}k−1 i=0 . ∞ {Kk }∞ k=0 family of Markov kernels, prior kernels, and {gk }k=0 densities w.r.t. a measure λ, local likelihood.
Aim: approximating filtering measure φk (u) , E [u(Xk )|Y0:k = y0:k ], for possibly several B(Xt+1 )/B(R)-measurable functions of interest u. ˜ Solution: SMC k , µ = φk+1 , and R with Ξ0 = Ξ = R, ν = φ Lk (ξ, A) = A gk+1 (ξ , Yk+1 ) Kk (ξ, dξ 0 ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Direct simulation from µN most often expensive (Accept/Reject, Hurzeler and Kunsch (1998), Kunsch (2005)). ˜N Solution: simulate new particles {ξ˜N,i }M i=1 from instrumental mixture MN
πN (·) ,
X i=1
ωN,i ψN,i R(ξN,i , ·) , PMN j=1 ωN,j ψN,j
where N {ψN,i = Ψ(ξN,i )}M i=1 positive numbers, adjustment multiplier weights,
R is a Markovian kernel, proposal kernel, ˜ importance weights {dµN /dπN (ξ˜N,i )}MN . i=1
Still, dµN /dπN expensive to evaluate: summing over MN terms.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j 2
Assign each draw (IN,i , ξ˜N,i ) the weight ω ˜ N,i , Φ(ξN,IN,i , ξ˜N,i ), where ˜ , Ψ(ξ)−1 Φ(ξ, ξ) second stage weights, proportional to
J. Cornebise, É. Moulines, J. Olsson
dL(ξ, ·) ˜ (ξ) , dR(ξ, ·)
dµaux N (IN,i , ξ˜N,i ). aux dπN
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Introduce stratum index as auxiliary variable, and target h i ˜ ωN,i L(ξN,i , Ξ) ˜ µaux L(ξN,i , A)/L(ξN,i , Ξ) N ({i} × A) , PMN ˜ j=1 ωN,j L(ξN,j , Ξ) ˜ on the product space {1, . . . , MN } × Ξ. 1
˜N Simulate {(IN,i , ξ˜N,i )}M i=1 from instrumental distribution
ωN,i ψN,i aux πN ({i} × A) , PM R(ξN,i , A) . N j=1 ωN,j ψN,j 2
Assign each draw (IN,i , ξ˜N,i ) the weight ω ˜ N,i , Φ(ξN,IN,i , ξ˜N,i ), where ˜ , Ψ(ξ)−1 Φ(ξ, ξ) second stage weights, proportional to
3
dL(ξ, ·) ˜ (ξ) , dR(ξ, ·)
dµaux N (IN,i , ξ˜N,i ). aux dπN ˜
N Marginalize: discard the indices and take {(ξ˜N,i , ω ˜ N,i )}M i=1 as an approximation of µ.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Skip Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
To beginning Courtesy of F. Campillo, INRIA J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Auxiliary Sampling Importance Sampling Illustration Adaptation
Adaptation: past and present
Aim: Ease the role of the user by automatically tuning the key parameters of the SMC algorithm. Past work: Important topic, previous approaches adapting: Number of particles: Legland and Oudjane (2006), and later Hu, Schon and Ljung (to appear), avoid degeneracy of the weights; also KLD-Sampling by Fox (2003), refined in Soto (2005) and Straka and Simandl (2006). Proposal kernel: Pitt and Shephard (1999) and Doucet, Godsill and Andrieu (2000), approximate so-called optimal kernel; Chan, Doucet and Tadic (2003) minimize expectation of a cost function, e.g. MSE or effective sample size over a parametric family of kernels. All rely on common criterions to assess the quality of the swarm. Sequential setting: function-free criterions required – due to recursivity of variance.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Definitions and criterions Asymptotics of criterions Optimal weights
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Common measures of discrepancy
Two probability measures µ and ν in P(Λ), s.t. µ ν. Recall the Kullback-Leibler Discrepancy (KLD) and Chi-Square Discrepancy (CSD): Definition: KLD and CSD Z dKL (µ||ν) ,
log ZΛ
dχ2 (µ||ν) ,
[ Λ
J. Cornebise, É. Moulines, J. Olsson
dµ (λ) µ(dλ) , dν
dµ (λ) − 1]2 ν(dλ) . dν
Adaptive refueling in SMC
Definitions and criterions Asymptotics of criterions Optimal weights
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Quality criterions aux Empirical estimates of the KLD and CSD between µaux and πN . N
Quality criterions
˜
˜N M
MN ˜ −1 E({˜ ωN,i }i=1 ),Ω N
X
“ ” ˜NΩ ˜ −1 ω ˜ N,i log M ˜ N,i , N ω
i=1
CV
2
˜N ({˜ ωN,i }M i=1 )
˜N M
˜NΩ ˜ −2 ,M N
X
2 ω ˜ N,i −1.
i=1
˜ N = PM˜ N ω where Ω i=1 ˜ N,i . ˜ N ). E: negated Shannon entropy of the importance weights (note the M CV2 : square of the widely used coefficient of variation of the importance weights, suggested by Kong, Liu and Wong (1994) and Liu and Chen (1995). Both minimal when all the weights are equal, maximal when all of the weights are zero except one. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
Two fundamental measures ˜ Define, for any A ∈ (B(Ξ) ⊗ B(Ξ)): ν⊗L (A) ˜ νL(Ξ) ν[Ψ] ⊗ R ∗ πΨ (A) , (A) ν(Ψ) µ∗ (A) ,
where the outer product of a measure and a kernel is defined as ZZ ν ⊗ L(A) , ν(dξ) L(ξ, dξ 0 )1A (ξ, ξ 0 ) , ˜ Ξ×Ξ
and a measure weighted by a function Ψ ∈ B(Ξ) is defined as ν[Ψ](B) , ν(Ψ1B ) , for B ∈ B(Ξ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We have the following limits, as N → ∞. Theorem Under weak technical conditions on the kernels and the weight function, 1
Asymptotic behavior of KLD: aux ∗ ∗ dKL (µaux N ||πN ) −→ dKL ( µ k πΨ ) , P
2
Asymptotic behavior of CSD: aux ∗ ∗ dχ2 (µaux N ||πN ) −→ dχ2 ( µ k πΨ ) . P
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We have the same limits for E and CV2 , as N → ∞. Theorem Under the same conditions, 1
Asymptotic behavior of KLD-based criterion: ˜
∗ ∗ N E({˜ ωN,i }M i=1 ) −→ dKL ( µ k πΨ ) , 2
P
Asymptotic behavior of CSD-based criterion: ˜
∗ ∗ N CV2 ({˜ ωN,i }M i=1 ) −→ dχ2 ( µ k πΨ ) .
J. Cornebise, É. Moulines, J. Olsson
P
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Definitions and criterions Asymptotics of criterions Optimal weights
Explicit forms for optimal weights
From the previous limiting expressions, we exhibit explicit forms for optimal adjustment multiplier weights. Corollary Under weak technical conditions, 1
The optimal weight function for KLD-based adaptation is aux ∗ ˜ arg min dKL (µaux N ||πN ) = ΨKL,R (ξ) , L(ξ, Ξ) . Ψ
2
The optimal weight function for CSD-based adaptation is sZ dL(ξ, ·) ˜ aux aux ∗ ˜ . (ξ) L(ξ, dξ) arg min dχ2 (µN ||πN ) = Ψχ2 ,R (ξ) , ˜ dR(ξ, ·) Ψ Ξ
KLD optimal weights independant of R. Special case: R ∝ L.
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
aux These insights lead to adaptive design of πN ,
Key idea Minimize the criterions over some parametric family {Rθ }θ∈Θ of proposal kernels. Leads to auxiliary proposal ωN,i ψN,i aux πN,θ ({i} × A) , PM Rθ (ξN,i , A) . N j=1 ωN,j ψN,j
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
aux These insights lead to adaptive design of πN ,
Key idea Minimize the criterions over some parametric family {Rθ }θ∈Θ of proposal kernels. Leads to auxiliary proposal ωN,i ψN,i aux πN,θ ({i} × A) , PM Rθ (ξN,i , A) . N j=1 ωN,j ψN,j Two approaches: 1
Minimize E and CV2 , fixed seeds (non detailed here).
2
Minimize MN aux dKL (µaux N ||πN,θ )
=
XZ i=1
log ˜ Ξ
! dµaux N ˜ ˜ (i, ξ) µaux N (i, dξ) . aux dπN,θ
via a Cross-Entropy approach: iterative Importance Sampling approximation. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Cross Entropy-based adaptation ` Iteration `, current fit θN ∈ Θ. 1
` aux ` ` )} from πN,θ IS approximation (E): Sample MN particles {(IN,i , ξ˜N,i ` .
2
Optimization (M): Compute exact solution to the approximation
N
˜` M N
`+1 θN
` Xω ˜ N,i log , arg min ˜ Ω` θ∈Θ i=1
N
! dµaux ` ` N (IN,i , ξ˜N,i ) , aux dπN,θ
or equivalently, in presence of densities, ˜` M N `+1 θN , arg max θ∈Θ
` Xω ˜ N,i ` log rθ (ξN,I ` , ξ˜N,i ). N,i ˜` Ω i=1
N
where rθ (ξ, ·) density of Rθ (ξ, ·). Choose family {Rθ } s.t. closed-form update: Normal Exponential Family. Theoretical justication in progress, close resemblance with Monte-Carlo EM algorithms. J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
We are here −→ •
1
Particle filtering: Introduction Auxiliary Sampling Importance Sampling Illustration Adaptation
2
Theory: quality criterions Definitions and criterions Asymptotics of criterions Optimal weights
3
Methodology: Adaptive importance sampling Optimizing criterions over a parametric family Benchmark on simulations Conclusion
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Benckmark: comparative study of the Mean-Square Error (MSE) on first order time-homogeneous autoregressions observed in noise: Reminder Xk+1 = m(Xk ) + σw (Xk )Wk+1 ,
k≥0,
Yk = Xk + σv Vk ,
For these very simple models, explicit forms for: ˜ – also optimal KLD weights for any Optimal weight Ψ∗ (ξ) , L(ξ, Ξ) kernel Ψ∗KL,R (ξ) ˜ Optimal kernel R∗ (ξ, ·) , L(ξ, ·)/L(ξ, Ξ) Optimal CSD weights for prior kernel Ψ∗χ2 ,K (ξ)
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
Explicit forms - for sake of curiosity Optimal weight Ψ∗ (ξ) Ψ∗ (ξ) = N (Yk+1 ; m(ξ),
p 2 (ξ) + σ 2 ) σw v
Optimal kernel R∗ (ξ, ·) has density r∗ (ξ, ξ 0 ) = N (x0 ; τ (ξ, Yk+1 ), η(ξ)) , with τ (ξ, Yk+1 ) ,
2 σw (ξ)Yk+1 + σv2 m(ξ) , 2 (ξ) + σ 2 σw v
η 2 (ξ) ,
2 σw (ξ)σv2 2 σw (ξ) + σv2
.
Important: known mode ! Optimal CSD weights for prior kernel Ψ∗χ2 ,K (ξ) s Ψ∗χ2 ,K (ξ)
∝
„ « 2 Yk+1 m(ξ) 2σv2 ×exp − 2 + 2 [2Yk+1 − m(ξ)] . 2 (ξ) + σ 2 2σw σv 2σw (ξ) + σv2 v
We recall that Ψ∗KL,K (ξ) = Ψ∗k (ξ). J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
CE-update formulae Intentionally simple example, adaptation over n o Rθ (ξ, ·) , N (τ (ξ, Yk+1 ), θ η(ξ)) : ξ ∈ R, θ > 0 . ∗ aux Hence known optimal value θN , arg minθ>0 dKL (µaux N ||πN,θ ) = 1. Pretend we ignore it. Available KLD: MN aux dKL (µaux N ||πN,θ ) =
X i=1
" × log
∗ ωN,i ψN,i PMN ∗ j=1 ωN,j ψN,j
!
∗ ψN,i ΩN
PMN
j=1
∗ ωN,j ψN,j
Closed-form update formula for CE: ( M` ` θN N X ω ˜ N,i `+1 θN = ` ˜ θN η 2 ` i=1 Ω N θ
` θN ξ˜N,i
1 + log θ + 2
«# 1 −1 , θ2
!2 )1/2 −τ
θ`
N IN,i
N IN,i
2 where τN,i , τ (ξN,i , Yk+1 ) and ηN,i , η 2 (ξN,i ). J. Cornebise, É. Moulines, J. Olsson
„
Adaptive refueling in SMC
,
Particle filtering: Introduction Theory: quality criterions Methodology: Adaptive importance sampling
Optimizing criterions over a parametric family Benchmark on simulations Conclusion
ARCH model Gaussian Auto-Regressive Conditional Heteroscedasticity model observed in p noise. mk (ξ) ≡ 0, σw (ξ) = β0 + β1 ξ 2 , i.e. q Xk+1 = β0 + β1 Xk2 Wk+1 , k ≥ 0 , Yk = Xk + σv Vk ,
k≥0.
1
Plain nonadaptive bootstrap particle filter (Ψ ≡ 1),
2
Optimal filter, i.e. optimal weights Ψ∗ and optimal kernel R∗ ,
3
Auxiliary filter with optimal χ2 weights Ψ∗χ2 ,K , prior kernel,
4
∗ Bootstrap filter with θN (known to be 1),
5
CE-based bootstrap filter (at last !).
J. Cornebise, É. Moulines, J. Olsson
Adaptive refueling in SMC
