Beating the Union Bound by Geometric Techniques Raghu Meka (IAS & DIMACS)

Union Bound

“

Popularized by Erdos

impossible, whatever remains, however improbable, must be the truth ”

Probabilistic Method 101 • Ramsey graphs – Erdos • Coding theory – Shannon • Metric embeddings – • … Johnson-Lindenstrauss

Beating the Union Bound • Not always enough Lovasz Local Lemma: 𝐸 1 , 𝐸 2 , … , 𝐸 𝑛 , 𝑑 Pr 𝐸 𝑖 dependent. < 1/4𝑑, ⇒ Pr ∪ 𝐸 𝑖 < 1.

• Constructive: Beck’91, …, Moser’09, …

Beating the Union Bound I. Optimal, explicit 𝜀 -nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method

Geometric techniques “Truly” constructive

Outline I. Optimal, explicit 𝜀 -nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method

Epsilon Nets • Discrete approximations • Applications: integration, comp. geometry, …

Epsilon Nets for Gaussians Discrete approximations of Gaussian Explicit Even existence not clear!

Nets in Gaussian space Thm: Explicit 𝜀 -net of size (1/𝜀) 𝑂 𝑘 .

• Optimal: Matching lower bound • Union bound: (𝑘/𝜀) 𝑘 • Dadusch-Vempala’12: ((log 𝑘)/𝜀) 𝑘

First: Application to Gaussian Processes and Cover Times 10

Gaussian Processes (GPs) Multivariate Gaussian Distribution

Supremum of Gaussian Processes (GPs) Given (𝑋 𝑖 ) want to study • Supremum is natural: eg., balls and bins

Supremum of Gaussian Processes (GPs) Given 𝑣 1 , … , 𝑣 𝑛 ∈ 𝑅 𝑑 , want to study log 𝑛 .

Why Gaussian Processes?

Stochastic Processes Functional analysis Convex Geometry Machine Learning Many more!

Cover times of Graphs Aldous-Fill 94: Compute cover time Eg: 𝑐𝑜𝑣𝑒𝑟 𝐾 𝑛 = Θ(𝑛 log 𝑛) • KKLV00: 𝑂((log log 𝑛) 2 approximation 2 𝑛) • Feige-Zeitouni’09: FPTAS for trees

Cover Times and GPs Thm (Ding, Lee, Peres 10): O(1) det. poly. time approximation for cover time.

Transfer to GPs Compute supremum of GP

Computing the Supremum Question (Lee10, Ding11): PTAS for computing the supremum of GPs?

• Ding, Lee, Peres 10: 𝑂(1) approximation • Can’t beat 𝑂(1) : Talagrand’s majorizing measures

Main Result Thm: PTAS for computing the supremum of Gaussian processes.

Thm: PTAS for computing cover time of bounded degree graphs.

Heart of PTAS: Epsilon net (Dimension reduction ala JL, use exp. size net)

Construction of Net 19

Construction of 𝜀 -net Simplest possible: univariate to multivariate 𝑘 𝑘 1. How fine a net?

Construction of 𝜀 -net Simplest possible: univariate to multivariate 𝑘 𝑘 Lem: Granularity 𝛿~𝑓(𝜀) enough.

Key point that beats union bound

Construction of 𝜀 -net This talk: Analyze ‘step-wise’ approximator Even out mass in interval [−𝛿, 𝛿] .

−4𝛿 −3𝛿 −2𝛿 𝛿 𝛾 ℓ 𝛿 2𝛿 3𝛿 4𝛿

Construction of 𝜀 -net Take univariate net and lift to multivariate 𝑘 𝑘 𝛾 −4𝛿 −3𝛿 −2𝛿 𝛿 𝛿 𝛾 ℓ 2𝛿 3𝛿 4𝛿 Lem: Granularity 𝛿~𝑓(𝜖) enough.

Dimension Free Error Bounds Thm: For 𝛿~ 𝜖 1.5

, 𝜑 a norm, 𝛾 • Proof by “sandwiching” • Exploit convexity critically −4𝛿 −3𝛿 −2𝛿 𝛿 𝛿 𝛾 ℓ 2𝛿 3𝛿 4𝛿

Analysis of Error Def: Sym. p, q.

∀ p ≼ 𝑞 (less peaked), if sym. convex sets K , 𝑝 𝐾 ≤ 𝑞 𝐾 .

• Why interesting? For any norm,

Analysis for Univarate Case Fact: 𝛾 ℓ ≼ 𝛾.

Proof: Spreading away from origin!

−4𝛿 −3𝛿 −2𝛿 𝛿 𝛿 2𝛿 3𝛿 4𝛿

Analysis for Univariate Case Fact: 𝑢 𝛾 ≼ 𝛾 𝑢 .

𝛾 ℓ .

Proof: For , 𝛿 ≪ 𝜖, , 𝛾 𝑢 = pdf of 𝑌 .

earlier spreading.

Push mass towards origin.

𝛾 𝑢

Analysis for Univariate Case Combining upper and lower: 𝛾 ℓ 𝛾 𝛾 𝑢

Lifting to Multivariate Case 𝑘 𝑘 𝑘 𝛾 ℓ 𝛾 𝑢 Kanter’s Lemma(77): and

unimodal,

Dimension free!

Lifting to Multivariate Case 𝑘 𝑘 𝑘 𝛾 ℓ 𝛾 𝛾 𝑢 Dimension free: key point that beats union bound!

Summary of Net Construction 1. Granularity 𝜀 1.5

enough 2. Cut points outside 𝑘 -ball Optimal 𝜀 -net

Outline I. Optimal, explicit 𝜀 -nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method

Discrepancy • Subsets 𝑆 1 , 𝑆 2 , … , 𝑆 𝑚 ⊆ [𝑛] • Color with 1 or 1 to minimize imbalance 3 1 1 0 1

Discrepancy Examples • Fundamental combinatorial concept Arithmetic Progressions Roth 64: Ω(𝑛 1/4 ) Matousek, Spencer 96: Θ(𝑛 1/4 )

Discrepancy Examples • Fundamental combinatorial concept Halfspaces Alexander 90: Matousek 95:

Why Discrepancy?

Complexity theory Communication Complexity Computational Geometry Pseudorandomness Many more!

Spencer’s Six Sigma Theorem “Six standard deviations suffice” Spencer 85: System with n sets has discrepancy at most . • Central result in discrepancy theory.

• Tight: Hadamard • Beats union bound :

A Conjecture and a Disproof Spencer 85: System with n sets has discrepancy at most . • Non-constructive pigeon-hole proof

Six Sigma Theorem New elementary geometric proof of Spencer’s result Main: Can efficiently find a coloring with discrepancy 𝑂 𝑛 .

constructive • Algorithmic partial coloring lemma • Extends to other settings

Outline of Algorithm 1. Partial coloring method 2. EDGE-WALK: geometric picture

Partial Coloring Method • Beck 80: find partial assignment with < 𝑛/2 zeros 0 1

Partial Coloring Method Input: Output: Lemma: Can do this in randomized time.

Outline of Algorithm 1. Partial coloring Method 2. EDGE-WALK: Geometric picture

Discrepancy: Geometric View • Subsets 𝑆 1 , 𝑆 2 , … , 𝑆 𝑚 ⊆ [𝑛] • Color with 1 or 1 to minimize imbalance 1 2 3 4 5

1 * 1 1 *

3

1 3 * 1 1 * 1

1

-1 1 1 1 1 1 1

1

1 1 * * * 1 1

0

1 0 1 * 1 * 1

1

-1 1

Discrepancy: Geometric View • Vectors 𝑣 1 , 𝑣 2 , … , 𝑣 𝑚 ∈ 0,1 𝑛 .

• Want 1 2 3 4 5

1 * 1 1 * 1 3 * 1 1 * 1 -1 1 1 1 1 1 1 1 1 * * * 1 1 1 0 1 * 1 * 1 -1 1

Discrepancy: Geometric View • Vectors 𝑣 1 , 𝑣 2 , … , 𝑣 𝑚 ∈ 0,1 𝑛 .

• Want Gluskin 88: Polytopes, Kanter’s lemma, ... !

Goal: Find non-zero lattice point inside

Edge-Walk • Start at origin • Brownian motion till you hit a face • Brownian motion within the face

Edge-Walk: Algorithm Gaussian random walk in subspaces • Subspace V, rate 𝛾 • Gaussian walk in V Standard normal in V: Orthonormal basis change

Edge-Walk Algorithm Discretization issues: hitting faces • Might not hit face • Slack: face hit if close to it.

Edge-Walk: Algorithm • Input: Vectors 𝑣 1 , 𝑣 2 , … , 𝑣 𝑚 .

• Parameters: 𝛿, Δ, 𝛾 ≪ 𝛿 , 𝑇 = 1/𝛾 2 1.

𝑋 0 = 0.

For 𝑡 = 1, … , 𝑇.

2.

𝑉𝑎𝑟 𝑡 = Cube faces nearly hit by 𝑋 𝑡 .

𝐷𝑖𝑠𝑐 𝑡 = Disc. faces nearly hit by 𝑋 𝑡 .

𝑉 𝑡 = Subspace orthogonal to 𝑉𝑎𝑟 𝑡 , 𝐷𝑖𝑠𝑐 𝑡

Edgewalk: Partial Coloring Lem: For with prob 0.1 and

Edgewalk: Analysis Discrepancy faces much farther than cube’s .

′ 𝑠] 1

Six Suffice 1. Edge-Walk: Algorithmic partial coloring 2. Recurse on unfixed variables Spencer’s Theorem

Summary I. Optimal, explicit 𝜀 -nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method

Geometric techniques

Others: Invariance principle for polytopes (Harsha, Klivans, M.’10), …

Open Problems • FPTAS for computing supremum?

• Beck-Fiala conjecture 81? – Discrepancy 𝑂( 𝑡) for degree 𝑡 .

• Applications of Edgewalk rounding?

Rothvoss’13: Improvements for bin-packing!

Thank you

Edgewalk Rounding Th: Given 𝑣 1 , … , 𝑣 𝑚 , thresholds 𝜆 1 , 𝜆 2 , … 𝜆 𝑚 , Can find 𝑋 ∈ −1,1 1.

2. 𝑛 with