A PTAS for Computing the Supremum of Gaussian Processes
A PTAS for Computing the Supremum of Gaussian Processes Raghu Meka (Microsoft Research)
Gaussian Processes (GPs) Multivariate Gaussian Distribution
Supremum of Gaussian Processes (GPs) Given ((𝑋↓𝑖 )) want to study
• Supremum is natural: eg., balls and bins
Supremum of Gaussian Processes (GPs) Given 𝑣↓1 ,…, 𝑣↓𝑛 ∈𝑅↑𝑑 , want to study
• Union bound: √log𝑛 .
𝑣↓2
𝑣↓1
When is the supremum smaller?
Cover times of Graphs
Aldous-Fill 94: Compute cover time Fundamental graph parameter deterministically?
Eg: 𝑐𝑜𝑣𝑒𝑟(𝐾↓𝑛 )=Θ(𝑛log𝑛) • KKLV00: 𝑂((loglog𝑛)𝑐𝑜𝑣𝑒𝑟(𝐺𝑟𝑖𝑑) ↑2 ) approximation =Θ(𝑛 • Feige-Zeitouni’09: FPTAS log↑2for 𝑛)trees
Cover Times and GPs Thm (Ding, Lee, Peres 10): O(1) det. poly. time approximation for cover time.
• Transfer to GPs Thm (DLP10): Winkler-Zuckerman “blanket-time” conjectures.
• Compute supremum of GP
Computing the Supremum Question (Lee10, Ding11): PTAS for computing the supremum of GPs?
• Ding, Lee, Peres 10: 𝑂(1) approximation – Fernique-Talagrand majorizing measures
Main Result Thm:Given A PTAS computing supremum of Thm: 𝑉⊆for 𝑅↑𝑑 , a 𝑑↑𝑂(1) the |𝑉|↑ 𝑂↓𝜖 (1) det. processes.
algorithm Gaussian to compute (1+𝜖) approx. to
Ding 12: PTAS for computing cover time of bounded degree graphs.
Outline of Algorithm 1. Dimension reduction – Slepian’s Lemma, Johnson-Lindenstrauss
2. Optimal eps-nets in Gaussian space – Kanter’s lemma, univariate to multivariate
Dimension Reduction Idea: JL projection, solve in projected space Use deterministic JL – EIO02, S02. V W
• 𝑉⊆𝑅↑𝑑 • 𝑊=𝑃𝑟𝑜𝑗(𝑉)⊆𝑅↑𝑘 , 𝑘=𝑂(log |𝑉|/𝜖↑2 ) .
Analysis: Slepian’s Lemma 𝑊=𝑃𝑟𝑜𝑗(𝑉)
Analysis: Slepian’s Lemma 𝑊=𝑃𝑟𝑜𝑗(𝑉)
• Enough to solve for W • Enough to be exp. in dimension
Outline of Algorithm 1. Dimension reduction – Slepian’s Lemma, Johnson-Lindenstrauss
2. Optimal eps-nets in Gaussian space – Kanter’s lemma, univariate to multivariate
Epsilon Nets • Discrete approximations • Applications: integration, comp. geometry, …
Epsilon Nets for Gaussians Discrete approximations of Gaussian Explicit
Nets in Gaussian space Thm: Explicit 𝜀-net of size (1/𝜀)↑𝑂(𝑘) .
• Optimal: Matching lower bound • Naïve bound: (√𝑘/𝜀)↑𝑘 • Dadush-Vempala’12: ((log𝑘)/𝜀)↑𝑘
Construction of Net
17
Construction of 𝜀-net Simplest possible: univariate to multivariate 𝑘
1. How fine a net?
𝑘.
2. Naïve: How big1/√ a net?
𝑘
Construction of 𝜀-net Simplest possible: univariate to multivariate 𝑘
Lem: Granularity 𝛿~𝑓(𝜀) enough.
𝑘
Construction of 𝜀-net This talk: Analyze ‘step-wise’ approximator Even out mass in interval [−𝛿, 𝛿].
−4𝛿−3𝛿−2𝛿 -𝛿 𝛿 𝛾↓ℓ𝓁
2𝛿 3𝛿 4𝛿
Construction of 𝜀-net Take univariate net and lift to multivariate 𝑘
𝛾
−4−3 𝛿 −2 𝛿 𝛿-𝛾𝛿↓𝛿 ℓ𝓁 2𝛿3𝛿4𝛿
Lem: Granularity 𝛿~𝑓(𝜖) enough.
𝑘
Dimension Free Error Bounds Thm: For 𝛿~ 𝜖↑1.5 , 𝜑 a norm,
• Proof by “sandwiching” • Exploit convexity critically 𝛾
−4−3 𝛿 −2 𝛿 𝛿-𝛾𝛿↓𝛿 ℓ𝓁 2𝛿3𝛿4𝛿
Analysis of Error Def: Sym. p, q. p≼𝑞 (less peaked), if ∀ sym. convex sets K, 𝑝(𝐾)≤𝑞(𝐾).
• Why interesting? For any norm,
Analysis for Univarate Case Fact: 𝛾↓ℓ𝓁 ≼𝛾. Spreading away from origin!
Proof:
−4𝛿−3𝛿−2𝛿 -𝛿
𝛿
2𝛿 3𝛿 4𝛿
Analysis for Univariate Case Def: down 𝛾↓ℓ𝓁 . Fact:𝛾↓𝑢 𝛾≼= scaled 𝛾↓𝑢 . 𝑋←𝛾For ↓ℓ𝓁 ,𝛿𝑌≪ =𝜖(1− 𝜖)𝑋, 𝛾push ↓𝑢 = pdf of 𝑌. Proof: , inward compensates earlier spreading.
𝛾↓𝑢
Push mass towards origin.
Analysis for Univariate Case Combining upper and lower:
𝛾↓ℓ𝓁
𝛾↓
𝛾↓𝑢 ↓
Lifting to Multivariate Case 𝑘
𝑘
𝛾↓ℓ𝓁
𝛾↓
𝑘
𝛾↓𝑢 ↓
Kanter’s and unimodal,
Key Lemma(77): for univariate: “peakedness”
Dimension free!
Lifting to Multivariate Case 𝑘
𝑘
𝛾↓ℓ𝓁
𝛾↓
Dimension free: key point that beats union bound!
𝑘
𝛾↓𝑢 ↓
Summary of Net Construction 1. Granularity 𝜀↑1.5 enough 2. Cut points outside √𝑘 -ball
Optimal 𝜀-net
Summary of Algorithm 1. Dimension reduction – Slepian’s Lemma
2. Optimal eps-nets for Gaussians – Kanter’s lemma
PTAS for Supremum
Open Problems Q: Lee, Peres): Conjecture (Ding, Asymptotically tight connection between GFF and cover time? • Implies a PTAS for cover time on all graphs.
Open Problems Slepian’s lemma for graphs? Graphs G, H on same set of vertices.
𝐻𝑖𝑡𝑇𝑖𝑚𝑒↓𝐺 (𝑢,𝑣)≤𝐻𝑖𝑡𝑇𝑖𝑚𝑒↓𝐻 (𝑢,𝑣), ∀𝑢,𝑣
⟹𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐺)≤𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐻)?
Ding, Lee, Peres 10:
𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐺)=𝑂(𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐻)).
Thank you
Gaussian Processes (GPs) Multivariate Gaussian Distribution
Supremum of Gaussian Processes (GPs) Given ((𝑋↓𝑖 )) want to study
• Supremum is natural: eg., balls and bins
Supremum of Gaussian Processes (GPs) Given 𝑣↓1 ,…, 𝑣↓𝑛 ∈𝑅↑𝑑 , want to study
• Union bound: √log𝑛 .
𝑣↓2
𝑣↓1
When is the supremum smaller?
Cover times of Graphs
Aldous-Fill 94: Compute cover time Fundamental graph parameter deterministically?
Eg: 𝑐𝑜𝑣𝑒𝑟(𝐾↓𝑛 )=Θ(𝑛log𝑛) • KKLV00: 𝑂((loglog𝑛)𝑐𝑜𝑣𝑒𝑟(𝐺𝑟𝑖𝑑) ↑2 ) approximation =Θ(𝑛 • Feige-Zeitouni’09: FPTAS log↑2for 𝑛)trees
Cover Times and GPs Thm (Ding, Lee, Peres 10): O(1) det. poly. time approximation for cover time.
• Transfer to GPs Thm (DLP10): Winkler-Zuckerman “blanket-time” conjectures.
• Compute supremum of GP
Computing the Supremum Question (Lee10, Ding11): PTAS for computing the supremum of GPs?
• Ding, Lee, Peres 10: 𝑂(1) approximation – Fernique-Talagrand majorizing measures
Main Result Thm:Given A PTAS computing supremum of Thm: 𝑉⊆for 𝑅↑𝑑 , a 𝑑↑𝑂(1) the |𝑉|↑ 𝑂↓𝜖 (1) det. processes.
algorithm Gaussian to compute (1+𝜖) approx. to
Ding 12: PTAS for computing cover time of bounded degree graphs.
Outline of Algorithm 1. Dimension reduction – Slepian’s Lemma, Johnson-Lindenstrauss
2. Optimal eps-nets in Gaussian space – Kanter’s lemma, univariate to multivariate
Dimension Reduction Idea: JL projection, solve in projected space Use deterministic JL – EIO02, S02. V W
• 𝑉⊆𝑅↑𝑑 • 𝑊=𝑃𝑟𝑜𝑗(𝑉)⊆𝑅↑𝑘 , 𝑘=𝑂(log |𝑉|/𝜖↑2 ) .
Analysis: Slepian’s Lemma 𝑊=𝑃𝑟𝑜𝑗(𝑉)
Analysis: Slepian’s Lemma 𝑊=𝑃𝑟𝑜𝑗(𝑉)
• Enough to solve for W • Enough to be exp. in dimension
Outline of Algorithm 1. Dimension reduction – Slepian’s Lemma, Johnson-Lindenstrauss
2. Optimal eps-nets in Gaussian space – Kanter’s lemma, univariate to multivariate
Epsilon Nets • Discrete approximations • Applications: integration, comp. geometry, …
Epsilon Nets for Gaussians Discrete approximations of Gaussian Explicit
Nets in Gaussian space Thm: Explicit 𝜀-net of size (1/𝜀)↑𝑂(𝑘) .
• Optimal: Matching lower bound • Naïve bound: (√𝑘/𝜀)↑𝑘 • Dadush-Vempala’12: ((log𝑘)/𝜀)↑𝑘
Construction of Net
17
Construction of 𝜀-net Simplest possible: univariate to multivariate 𝑘
1. How fine a net?
𝑘.
2. Naïve: How big1/√ a net?
𝑘
Construction of 𝜀-net Simplest possible: univariate to multivariate 𝑘
Lem: Granularity 𝛿~𝑓(𝜀) enough.
𝑘
Construction of 𝜀-net This talk: Analyze ‘step-wise’ approximator Even out mass in interval [−𝛿, 𝛿].
−4𝛿−3𝛿−2𝛿 -𝛿 𝛿 𝛾↓ℓ𝓁
2𝛿 3𝛿 4𝛿
Construction of 𝜀-net Take univariate net and lift to multivariate 𝑘
𝛾
−4−3 𝛿 −2 𝛿 𝛿-𝛾𝛿↓𝛿 ℓ𝓁 2𝛿3𝛿4𝛿
Lem: Granularity 𝛿~𝑓(𝜖) enough.
𝑘
Dimension Free Error Bounds Thm: For 𝛿~ 𝜖↑1.5 , 𝜑 a norm,
• Proof by “sandwiching” • Exploit convexity critically 𝛾
−4−3 𝛿 −2 𝛿 𝛿-𝛾𝛿↓𝛿 ℓ𝓁 2𝛿3𝛿4𝛿
Analysis of Error Def: Sym. p, q. p≼𝑞 (less peaked), if ∀ sym. convex sets K, 𝑝(𝐾)≤𝑞(𝐾).
• Why interesting? For any norm,
Analysis for Univarate Case Fact: 𝛾↓ℓ𝓁 ≼𝛾. Spreading away from origin!
Proof:
−4𝛿−3𝛿−2𝛿 -𝛿
𝛿
2𝛿 3𝛿 4𝛿
Analysis for Univariate Case Def: down 𝛾↓ℓ𝓁 . Fact:𝛾↓𝑢 𝛾≼= scaled 𝛾↓𝑢 . 𝑋←𝛾For ↓ℓ𝓁 ,𝛿𝑌≪ =𝜖(1− 𝜖)𝑋, 𝛾push ↓𝑢 = pdf of 𝑌. Proof: , inward compensates earlier spreading.
𝛾↓𝑢
Push mass towards origin.
Analysis for Univariate Case Combining upper and lower:
𝛾↓ℓ𝓁
𝛾↓
𝛾↓𝑢 ↓
Lifting to Multivariate Case 𝑘
𝑘
𝛾↓ℓ𝓁
𝛾↓
𝑘
𝛾↓𝑢 ↓
Kanter’s and unimodal,
Key Lemma(77): for univariate: “peakedness”
Dimension free!
Lifting to Multivariate Case 𝑘
𝑘
𝛾↓ℓ𝓁
𝛾↓
Dimension free: key point that beats union bound!
𝑘
𝛾↓𝑢 ↓
Summary of Net Construction 1. Granularity 𝜀↑1.5 enough 2. Cut points outside √𝑘 -ball
Optimal 𝜀-net
Summary of Algorithm 1. Dimension reduction – Slepian’s Lemma
2. Optimal eps-nets for Gaussians – Kanter’s lemma
PTAS for Supremum
Open Problems Q: Lee, Peres): Conjecture (Ding, Asymptotically tight connection between GFF and cover time? • Implies a PTAS for cover time on all graphs.
Open Problems Slepian’s lemma for graphs? Graphs G, H on same set of vertices.
𝐻𝑖𝑡𝑇𝑖𝑚𝑒↓𝐺 (𝑢,𝑣)≤𝐻𝑖𝑡𝑇𝑖𝑚𝑒↓𝐻 (𝑢,𝑣), ∀𝑢,𝑣
⟹𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐺)≤𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐻)?
Ding, Lee, Peres 10:
𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐺)=𝑂(𝐶𝑜𝑣𝑒𝑟𝑇𝑖𝑚𝑒(𝐻)).
Thank you
