poster template - Semantic Scholar

A Deterministic Analysis of Noisy Sparse Subspace Clustering for Dimensionality-reduced Data Yining Wang, Yu-Xiang Wang and Aarti Singh Carnegie Mellon University, Machine Learning Department

Subspace clustering: clustering data points into union of low-dimensional subspaces

Sparse Subspace Clustering (SSC, Elhamifar & Vidal 2007): state-of-the-art subspace clustering algorithm based on ℓ1 self-expression Step 1. Instance-level ℓ1 self-regression 2 𝑐𝑖 = argmin𝑐∈𝑅𝑁−1 𝑥𝑖 − 𝑐𝑋−𝑖 2 + 𝜆 𝑐 1 𝑁×𝑁 Step 2. Build similarity graph 𝐺 ∈ 𝑅 by taking 𝐺𝑖𝑗 = 𝑐𝑖𝑗 + 𝑐𝑗𝑖

Deterministic analysis of Noisy Sparse Subspace Clustering under dimension reduction Subspace incoherence: for subspace 𝑆ℓ define 𝑇 ℓ 𝜇ℓ = max(ℓ) 𝑉 𝑥 ∞ ℓ {normalize(𝑃𝑆_ℓ [𝑣(𝑥𝑖 𝑥∈𝑋\X

ℓ

where 𝑉 = )])} and 𝑣(𝑥) is the optimal solution to dual problem 2 𝑇 max𝑑 𝜈, 𝑥 + 0.5𝜆 𝜈 2 , 𝑠. 𝑡. 𝑋 𝜈 ∞ ≤ 1 𝜈∈𝑅

Mathematically: given 𝑥1 , ⋯ , 𝑥𝑁 ∈ 𝑑 𝑅 , find linear subspaces 𝑆1 , ⋯ , 𝑆𝐿 of dimension 𝑟 ≪ 𝑑 such that each 𝑥𝑖 approximately lies in some 𝑆𝑘 Applications: motion segmentation

Inradius: 𝜌ℓ characterizing inner-subspace data distribution

Step 3. Spectral clustering on similarity graph 𝐺 Question: will SSC still succeed if the ambient data dimension 𝒅 is reduced to 𝒑 ≪ 𝒅 by linear dimensionality reduction? 𝒑×𝒅 𝑿 = 𝚿𝐗, 𝚿∈𝑹 Motivation: computational efficiency, compressed measurement, missing data, data privacy, etc.

… and many more: face clustering, network hop counting, social graph mining, recommendation systems …

Method: Gaussian projection, Fast JohnsonLindenstrauss transform (FJLT), uniform row sampling, sketching, etc.

Property: subspace embedding property 2 2 Pr ∀𝒙 ∈ 𝑺, Ψ𝑥 2 ∈ 1 ± 𝜖 𝑥 2 ≥ 1 − 𝛿

No false connection: 𝑥𝑖 , 𝑥𝑗 ∈ 𝐸 𝐺 ⟹ 𝑥𝑖 , 𝑥𝑗 belong to the same cluster (subspace).

Main Theorem Let 𝜼 be the level of adversarial noise, 𝝐 be the parameter in subspace embedding property and 𝚫 = 𝐦𝐢𝐧ℓ (𝝆ℓ − 𝝁ℓ ) be the geometric gap. Then 𝑮 has no false connections with high probability if 𝝐 ≤ 𝐦𝐢𝐧

𝟏 𝚫 𝝀 , , 𝟑 𝟒(𝟐+𝝆) 𝟖

𝒄𝟐

𝟐 𝚫

−

𝟓𝜼𝟐 𝝆

− 𝟑𝜼