Gradient Estimation Revitalized - VISAGG

Gradient Estimation Revitalized Usman R. Alim1

¨ 1 Torsten Moller

Laurent Condat2

1 Graphics,

Usability, and Visualization (GrUVi) Lab. School of Computing Science Simon Fraser University 2 GREYC

Lab. Image Team Caen, France

gruvi

graphics + usability + visualization

gruvi

Motivation

graphics + usability + visualization

Good renderings need good gradients

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

2/22

gruvi

Motivation

graphics + usability + visualization

Good renderings need good gradients

Finite Differencing

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

2/22

gruvi

Motivation

graphics + usability + visualization

Good renderings need good gradients

Orthogonal Projection

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

2/22

Related Work: Quantitative Analysis

gruvi

graphics + usability + visualization

Quantitative Fourier analysis of scalar reconstruction schemes [Unser and Blu ’99] Linear interpolation revitalized [Blu et al. ’04] ¨ Extension to derivatives in 1D [Condat and Moller ’09]

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

3/22

gruvi

Overview (R3 )

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

Alim et al., GrUVi, GREYC

p

ϕ(x)

fapp (x)

prefilter

Gradient Estimation Revitalized

4/22

gruvi

Overview (R3 )

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p

ϕ(x)

fapp (x)

prefilter

Why prefilter? 1 Ensures approximation and original function agree at the lattice sites 2

Exploits the full approximation power of ϕ

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

4/22

gruvi

Overview (R3 )

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient estimation d3

ϕ(x − l3 /2)

l3 fapp

li : Principal lattice directions

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

4/22

gruvi

Overview (R3 )

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient estimation d3

ϕ(x − l3 /2)

l3 fapp

li : Principal lattice directions More general case considered in the paper

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

4/22

gruvi

Principal Directions

2D Cartesian: 2

Alim et al., GrUVi, GREYC

graphics + usability + visualization

Hexagonal: 3

Gradient Estimation Revitalized

5/22

gruvi

Principal Directions

graphics + usability + visualization

2D Cartesian: 2

Hexagonal: 3

CC: 3

BCC: 4

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

5/22

gruvi

Formal Description

graphics + usability + visualization

Approximate derivatives in the principal directions li Interested in a digital filter that approximates in the shift-invariant space V(Lh , ϕi ), i.e. li ∂li f (x) ≈ fapp (x) =

X 1 (f ∗ p ∗ di )[k]ϕih,k (x) h s

k∈Z

ϕi (x) := ϕ(x −

li x ) and ϕih,k (x) := ϕi ( − Lk) 2 h

l k n The filter di should be chosen so that k∂l f − fapp L2 = O(h ) where n is the approximation order of ϕ.

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

6/22

gruvi

Formal Description

graphics + usability + visualization

Approximate derivatives in the principal directions li Interested in a digital filter that approximates in the shift-invariant space V(Lh , ϕi ), i.e. li ∂li f (x) ≈ fapp (x) =

X 1 (f ∗ p ∗ di )[k]ϕih,k (x) h s

k∈Z

ϕi (x) := ϕ(x −

li x ) and ϕih,k (x) := ϕi ( − Lk) 2 h

l k n The filter di should be chosen so that k∂l f − fapp L2 = O(h ) where n is the approximation order of ϕ.

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

6/22

gruvi

Formal Description

graphics + usability + visualization

Approximate derivatives in the principal directions li Interested in a digital filter that approximates in the shift-invariant space V(Lh , ϕi ), i.e. li ∂li f (x) ≈ fapp (x) =

X 1 (f ∗ p ∗ di )[k]ϕih,k (x) h s

k∈Z

ϕi (x) := ϕ(x −

li x ) and ϕih,k (x) := ϕi ( − Lk) 2 h

l k n The filter di should be chosen so that k∂l f − fapp L2 = O(h ) where n is the approximation order of ϕ.

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

6/22

gruvi

Revitalization

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient estimation d3

Alim et al., GrUVi, GREYC

ϕ(x − l3 /2)

l3 fapp

Gradient Estimation Revitalized

7/22

gruvi

Shift Example (Cubic B-spline)

graphics + usability + visualization

0.6 0.5 0.4 0.3 0.2 0.1

-3

-2

1

-1

2

3

ϕ(x)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

8/22

gruvi

Shift Example (Cubic B-spline)

graphics + usability + visualization

0.6 0.5 0.4 0.3 0.2 0.1

-3

-2

1

-1

2

3

ϕ(x − k)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

8/22

gruvi

Shift Example (Cubic B-spline)

graphics + usability + visualization

0.6 0.5 0.4 0.3 0.2 0.1

-3

-2

1

-1

2

3

1

ϕ (x)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

8/22

gruvi

Shift Example (Cubic B-spline)

graphics + usability + visualization

0.6 0.5 0.4 0.3 0.2 0.1

-3

-2

1

-1

2

3

1

ϕ (x − k)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

8/22

gruvi

Why shift? (1D)

graphics + usability + visualization

1.0 0.7 0.6 0.5 0.5 0.4 -3

-2

1

-1

2

3

0.3 0.2

-0.5

0.1 -1.0 -2

-1

1

β2 (x) - Quadratic B-spline

2

β20 (x) = β1 (x + 21 ) − β1 (x − 12 ) (blue) V(Z, β1 (x)) can’t recover the exact derivative (purple) V(Z, β1 (x − 1/2)) can! (blue)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

9/22

gruvi

Why shift? (1D)

graphics + usability + visualization

1.0 0.7 0.6 0.5 0.5 0.4 -3

-2

1

-1

2

3

0.3 0.2

-0.5

0.1 -1.0 -2

-1

1

β2 (x) - Quadratic B-spline

2

β20 (x) = β1 (x + 21 ) − β1 (x − 12 ) (blue) V(Z, β1 (x)) can’t recover the exact derivative (purple) V(Z, β1 (x − 1/2)) can! (blue)

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

9/22

How to shift in higher dimensions?

gruvi

graphics + usability + visualization

Directional derivative of a 4th order hexagonal box spline is a linear combination of two lower order shifted box splines Shifts are in the direction of the derivative 1

Choose s linearly independent principal directions

2

Shift the symmetric box spline along those principal directions

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

10/22

How to shift in higher dimensions?

gruvi

graphics + usability + visualization

Directional derivative of a 4th order hexagonal box spline is a linear combination of two lower order shifted box splines Shifts are in the direction of the derivative 1

Choose s linearly independent principal directions

2

Shift the symmetric box spline along those principal directions

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

10/22

gruvi

Error Analysis

graphics + usability + visualization

How to predict the directional derivative error, given an approximation space V(L, ψ) and a filter r

Error Kernel b 2 ψ(ω)

2 R(ω) b ? b ˚ (ω) bψ (ω) ψ E (ω) := 1 − − +A bψ (ω) jlT ω A {z } | | {z } l

Emin (ω)

l (ω) Eres

b ↔ r: Filter applied to samples l: A principal lattice direction R bψ : Autocorrelation sequence A

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

11/22

gruvi

Filter Design

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient estimation d3

ϕ(x − l3 /2)

l3 fapp

Combined directional derivative filter ri = (p ∗ di )

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

12/22

gruvi

Asymptotic Optimality

graphics + usability + visualization

b 2 ψ(ω)

2 R(ω) b ? b ˚ (ω) bψ (ω) E l (ω) := 1 − ψ +A − bψ (ω) jlT ω A | {z } | {z } Emin (ω)

l (ω) Eres

For a minimum error approximation: l Eres (ω) = 0, not realizable!

l (ω) (as h → 0) Choose r so that Emin (ω) ∼ Eres

Plug-in our basis function ϕi and combined filter ri = p ∗ di

Optimality Criterion ci = jli T ω exp( j li T ω) + O(|ω|n+1 ) di ↔ D 2 No dependence on ϕ

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

13/22

gruvi

Asymptotic Optimality

graphics + usability + visualization

b 2 ψ(ω)

2 R(ω) b ? b ˚ (ω) bψ (ω) E l (ω) := 1 − ψ +A − bψ (ω) jlT ω A | {z } | {z } Emin (ω)

l (ω) Eres

For a minimum error approximation: l Eres (ω) = 0, not realizable!

l (ω) (as h → 0) Choose r so that Emin (ω) ∼ Eres

Plug-in our basis function ϕi and combined filter ri = p ∗ di

Optimality Criterion ci = jli T ω exp( j li T ω) + O(|ω|n+1 ) di ↔ D 2 No dependence on ϕ

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

13/22

gruvi

Asymptotic Optimality

graphics + usability + visualization

b 2 ψ(ω)

2 R(ω) b ? b ˚ (ω) bψ (ω) E l (ω) := 1 − ψ +A − bψ (ω) jlT ω A | {z } | {z } Emin (ω)

l (ω) Eres

For a minimum error approximation: l Eres (ω) = 0, not realizable!

l (ω) (as h → 0) Choose r so that Emin (ω) ∼ Eres

Plug-in our basis function ϕi and combined filter ri = p ∗ di

Optimality Criterion ci = jli T ω exp( j li T ω) + O(|ω|n+1 ) di ↔ D 2 No dependence on ϕ

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

13/22

Fourth-order 1D Filters di

gruvi

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

-3

-2

-1

Centered (pFIR):

Alim et al., GrUVi, GREYC

1

2

3

-3

1 2 1 [− 12 , 3 , 0, − 23 , 12 ]

-2

-1

Shifted (pFIR-s):

Gradient Estimation Revitalized

graphics + usability + visualization

1

2

3

1 9 1 [− 24 , 8 , − 98 , 24 , 0]

14/22

Combined Filter (p ∗ d) Comparison

gruvi

graphics + usability + visualization

Cubic B-spline 1.5

Derivative Error Kernel

Emin -cubic Emin -quadratic CC CC-s pFIR pFIR-s pANALYTIC

1

0.5

0

π 2

π

3π 2

ω Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

2π

and the de bic B-splin β3 (x − 12 ) For the equating T explained i while the o scalar prefi As show to better er filter CC-s spline whi the use of tent. Using pact on the error kerne response m Finally, the scalar kernel for t to the15/22OP

gruvi

Gradient Reconstruction

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient rec. d3

ϕ(x − l3 /2)

l3 fapp

Simple linear transformation li : Dual of li

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

16/22

gruvi

Results

graphics + usability + visualization

Tricubic B-splines on the Cartesian Cubic (CC) lattice Quintic Box spline on the Body-Centered Cubic (BCC) lattice [Entezari et al. ’08]

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

17/22

gruvi

Quantitative Comparison

pFIR, 24.14◦ , 2.11

pFIR-s, 11.53◦ , 1.15

P-OPT26, 18.10◦ , 1.96

P-FIR-s, 8.7◦ , 1.03

graphics + usability + visualization

Mean angular and magnitude errors are indicated Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

18/22

gruvi

Qualitative Comparison

graphics + usability + visualization

Centered vs. Shifted

pFIR

Alim et al., GrUVi, GREYC

P-OPT26

Gradient Estimation Revitalized

19/22

gruvi

Qualitative Comparison

graphics + usability + visualization

Centered vs. Shifted

pFIR-s

Alim et al., GrUVi, GREYC

P-FIR-s

Gradient Estimation Revitalized

19/22

gruvi

Qualitative Comparison

graphics + usability + visualization

DVR Centered vs. Shifted

pFIR Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

20/22

gruvi

Qualitative Comparison

graphics + usability + visualization

DVR Centered vs. Shifted

pFIR-s Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

20/22

gruvi

Conclusion

graphics + usability + visualization

scalar reconstruction f (x)

f [k]

sampling on L

p prefilter

fapp (x)

ϕ(x) dir. der. filter

reconstruction

d1

ϕ(x − l1 /2)

d2

ϕ(x − l2 /2)

d3

ϕ(x − l3 /2)

l1 fapp

l2 fapp

li i i fapp l

P

(∇f )app (x)

gradient estimation l3 fapp

Contributions Error Kernel to quantify accuracy of gradient estimation Two frameworks for designing asymptotically optimal derivative filters Shifted interpolation function → Better quality at no additional cost!

Alim et al., GrUVi, GREYC

Gradient Estimation Revitalized

21/22

Acknowledgements

gruvi

graphics + usability + visualization

Thank you for your attention Contact: [email protected] Source code is available at: http://www.cs.sfu.ca/~ualim/personal/research.html