Functional and Structural Implications of Non-Separability of Spectral and Temporal Responses in AI Jonathan Z. Simon David J. Klein Didier A. Depireux Shihab A. Shamma Institute for Systems Research and Dept. of Electrical & Computer Engineering University of Maryland Supported in part by MURI # N00014-97-1-0501 from the Office of Naval Research, # NIDCD T32 DC00046-01 from the National Institute on Deafness and Other Communication Disorders, # NSFD CD8803012 from the National Science Foundation. This poster is available at . Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
Introduction • We measure the response of cells in ferret Primary Auditory Cortex (AI) to dynamic, broadband sounds.
• The dynamic, broadband sounds are simple combinations of spectro-temporal basis functions, called “moving ripples.”
• By correlating the response with the stimulus, we derive Spectro-Temporal Receptive Fields (STRFs), a linear, quantitative descriptor of how a cell responds to dynamic sounds.
• The STRFs exhibit symmetries and patterns such as separability, and its generalization, quadrant separability.
• Quadrant separability does not arise from most neural networks, and can be used to rule out some models of neural connectivity.
Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
Summary • The STRFs measured for all units in AI are quadrant separable or fully separable.
• Quadrant separability is incompatible with simple summing of independent, fully separable sources.
• Summing two fully separable STRFs is quadrant separable if the temporal processing is in quadrature.
• There are three ways in which a fully separable STRF can become quadrant separable: power asymmetry, spectral asymmetry, & temporal asymmetry.
• Only power and spectral asymmetry contribute to quadrant separability in AI, not temporal.
• Quadrant Separability is incompatible with velocity selectivity. • Quadrant Separability and persistence of temporal symmetry strongly constrain possible models of neural connectivity. Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
STRFs in AI ... or by the (Fourier domain) ripple Transfer Function (TF).
Cells are characterized their SpectroTemporal Response Field (STRF)... STRF(t,x)
Ω
X
(cycles/octave)
|TF(w,Ω)|
x = log2 f/fo (octaves)
|F { }| 226/24a06.a1
0
t (ms)
0
-w
T
0
w
(cycles/second)
Moving ripples form the basis for the Fourier domain description of dynamic spectra. At time t and frequency x, the amplitude S(t,x) is given by: t (ms)
Ω (cycles/octave)
250
x (octaves)
0 5
S(t,x)= sin(2πwt + 2πΩx + Φ)
0.6
x = log2[f / f0] w = ripple velocity, modulation rate Ω = ripple frequency, spectral density
0
0.2 -12 Center for Auditory and Acoustic Research
-4
4
12
w (Hz) Institute for Systems Research University of Maryland
Temporally Orthogonal Ripple Combinations • STRF measured by reverse-correlating with dynamic spectrum of a broad-band stimulus.
• Temporally Orthogonal Ripple Combinations composed of ripples with different modulation rates. S(t,x) 5 octaves
stimulus
• Allow clean STRF estimates in relatively short time.
250 ms
response
100
R(t) (spikes/ sec)
0
-100 250 ms
Ω
The stimuli shown contain ripples covering the same range of ripple velocities, but at different ripple frequencies. w Center for Auditory and Acoustic Research
...
... 0
w Institute for Systems Research University of Maryland
Fully Separable STRF |TF(w,Ω)|
STRF(t,x)
x = log2[f / f0]
4k Hz
Q2
1.8 cyc/oct
Q1
|F { }|
g(x) 125 Hz
-32 Hz
226/20a.a1
0 Q1 Q2
250 ms
32 Hz
The STRF and TF are a product of a single spectral response function with a single temporal response function.
f(t) t
Shown above are the impulse responses f(t) and receptive fields g(x) derived from quadrant 1 (black) and quadrant 2 (red) of the transfer function by inverse Fourier transformation. STRF(t,x) = f(t) g(x)
f(t)
|F { }|
F(w)
TF(w,Ω) = F(w) G(Ω)
g(x) |F { }| G(Ω)
Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
Quadrant Separable STRF STRF(t,x)
x = log2[f / f0]
|TF(w,Ω)|
8 kHz
1.8 cyc/oct
|F { }| gi(x)
250 Hz
-32 Hz 0
250 ms
Q1 Q2
fi(t) t
This neuron responded twice as strong to rising frequencies than it did to falling frequencies.
fi(t)
|F { }|
Fi(w)
gi(x) |F { }| Gi(Ω)
Center for Auditory and Acoustic Research
32 Hz
The STRF is not separable, but each quadrant of the transfer function is, i.e., there are different spectral and temporal responses for upwards and downwards frequency modulation.
F1 ( w ) G1 (Ω) T ( w, Ω) = F2 ( w ) G2 (Ω)
w > 0, Ω > 0 w < 0, Ω > 0
and for Ω > 0: T(w,Ω) = T*(-w,-Ω)
Institute for Systems Research University of Maryland
Examples & Counterexamples Fully Separable
Ω
|F { }|
-w
Quadrant Separable
Ω
w
|F { }|
-w Center for Auditory and Acoustic Research
w
|F { }|
-w
Velocity Selective is Inseparable
Ω
Quadrant separability is incompatible with velocity selectivity.
w Institute for Systems Research University of Maryland
A Counterexample Fully Separable
Ω
|F { }|
-w
Fully Separable (displaced)
Ω
w
|F { }|
-w Center for Auditory and Acoustic Research
w
|F { }|
-w
Sum of two Fully Separable is Inseparable
Ω
Naive sum of two fully separable STRFs is inseparable.
w Institute for Systems Research University of Maryland
An Example Fully Separable
Ω
|F { }|
-w
Same Fully Separable but Lagged (& shifted spectrally)
Ω
w
|F { }|
-w Center for Auditory and Acoustic Research
w
|F { }|
-w
Sum of Non-Lagged and Lagged is Separable
Ω
Sum of two fully separable STRFs is separable if the temporal processing is in quadrature.
w Institute for Systems Research University of Maryland
Measuring Separability with SVD • Singular Value Decomposition (SVD) can be used to estimate the separability of a Transfer Function (possibly corrupted by noise). It decomposes the Transfer Function into a sum of Quadrant Separable Transfer Functions, ordered by their power.
• We apply SVD to each quadrant of the transfer function. Below, an STRF and the three most significant quadrant-separable components, derived from SVD: 1st Quadrant-separable Component
Raw Estimate Ω
+
= w T ( w, Ω)
=
2nd
+ ...
+
F11 ( w ) G11 (Ω)
+
F12 ( w ) G12 (Ω)
+
F13 ( w) G13 (Ω)
+K
w > 0, Ω > 0
F ( w ) G (Ω )
+
F ( w ) G (Ω )
+
F ( w ) G (Ω )
+K
w < 0, Ω > 0
1 2
1 2
2 2
2 2
3 2
3 2
x
=
+
+
+ ...
t
Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
SVD Example Q1
Fraction of total power
Q2
λi
0.5 0.5
0
1
2
3
4
5
6
7
8
9 10
0
1
2
3
Singular Value Number
4
5
6
7
8
9 10
Bootstrap estimate of noise power
• SVD naturally picks out high SNR components of a matrix. • Large jumps in the singular values separate signal from noise. • Jumps straddle bootstrap estimate of noise. • Noise can be removed by discarding lower-magnitude components. • All cells (31) measured in AI have a single dominant SVD component in each quadrant.
All units measured in AI are quadrant separable (or fully separable). Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
Measure of Inseparability • SVD supplies a natural measure of inseparability, αSVD α SVD
λ21 = 1 − 2 λ ∑i i
• αSVD ≈ 0 is fully separable • αSVD > 0.3 is strongly inseparable Population Statistics
αSVD = 0.35 Frequency
Frequency
αSVD = 0.06 10 5
225/33a06.m1 225/33a05.m2
218/15b06.m1 218/15b04.m2
0 Time
Center for Auditory and Acoustic Research
0
0.25
0.5
Time
Institute for Systems Research University of Maryland
Symmetry by Power • αd: Power asymmetry breaks full separability, producing quadrant separability αd = (P1 - P2)/(P1 + P2) P1 = (Power in quadrant 1) = (λ1)2 P2 = (Power in quadrant 2) = (λ2)2
• αd ≈ 0 is symmetric in power • |αd | > 0.3 is quite asymmetric in power—strongly inseparable Example STRF Frequency
αd = .79
Population Statistics αd
20
Contribution to αSVD
0.5
αSVD
10 229/11a05.m1 229/11a04.m2
Time Center for Auditory and Acoustic Research
0 -1
0 0
1
0
.51 correlation 0.5 |αd| 1 Institute for Systems Research University of Maryland
Spectral Symmetry • αs: Asymmetry between spectral cross-sections Gi(Ω): αs = 1 −
• αs • αs
where the quantity inside the big absolute value bars is the (complex) correlation between G1(Ω) and G2(Ω)
∑Ω > 0 G1 (Ω) G2* (Ω)
∑Ω > 0 G1 (Ω) G2 (Ω) 2
2
≈ 0 is spectrally symmetric > 0.3 is spectrally asymmetric—strongly inseparable
Frequency
Example STRF αs = .65
Population Statistics αs 20
0.5
Contribution to αSVD
αSVD
10 223/12a06.m1 223/12a05.m2
Time
Center for Auditory and Acoustic Research
0
0
0.5
1
0
0
.75 correlation 0.5 αs 1
Institute for Systems Research University of Maryland
Temporal Symmetry • αt: Asymmetry between temporal cross-sections Fi(w): αt = 1 −
• αt • αt
∑w>0 F1 (w) F2 (− w)
∑
F ( w ) F2 ( − w ) w>0 1 2
2
≈ 0 is temporally symmetric
where the quantity inside the big absolute value bars is the (complex) correlation between F1(w) and F2*(-w)
> 0.3 is temporally asymmetric—strongly inseparable
Frequency
Example STRF αt = .30
Population Statistics αt
20
Contribution to αSVD
0.5
αSVD
10 219/25b05.m1 219/25b04.m2
Time
0
.66 correlation 0.5 αt 1
0 0
0.5
1
0
Distribution is strongly skewed toward temporal symmetry. Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland
Symmetry Correlations • Mean of 3 separate symmetry
0.5
measures correlates well with full separability index.
.90 correlation 0.4 αSVD 0.3
0.2
0.1
00
0.1
0.2 0.3 mean(ασ, ατ, |αd|)
0.4
0.5
• Individual indices only partially correlated with each other. αt
1
.08 correlation
0.5 0
Center for Auditory and Acoustic Research
0
1 |αd|
.21 correlation
0.5
0.5
αs
1
0
1 |αd|
.27 correlation
0.5
0
0.5 α
1 s
0
0
0.5
αt 1
Institute for Systems Research University of Maryland
Models fully separable
Not Quadrant Separable
MGB AI
fully separable
fully separable
MGB fully separable
poral m e t e s am gged a l t u b
Center for Auditory and Acoustic Research
on functi
Quadrant Separable AI
Institute for Systems Research University of Maryland
Example STRF Recording Pairs 4000
αSVD = 12% αd = 15% αs = 4% αt = 7%
Frequency (Hz)
2000 1000 500 250
αSVD = 19% αd = 12% αs = 9% αt = 9%
2000 1000 500
αSVD = αd = αs = αt =
2000 1000 500
5% 0% 3% 1%
4000 2000 1000 500
235/16a03.a1e.3
125
250
4000
8000
αSVD = 47% αd = 52% αs = 26% αt = 13%
2000 1000 500
250 234/17a03.a1e.2
100 150 Time (ms)
200
250
2000 1000 500
αcorr = t
[ [
4% 3% 2% 4%
Center for Auditory and Acoustic Research
234/23c02.a1.2
50
100 150 Time (ms)
200
250 250
Different, Upper fully separable, Lower quadrant separable
Very similar, Both fully separable ε = ( 20%, 15%) αscorr =
125 0
] ]
6% 4% 7% 5%
αSVD = 77% αd = 0% αs = 26% αt = 13%
4000
250
50
αSVD = 55% αd = 26% αs = 9% αt = 9%
234/23c02.a1e.1
234/17a03.a1e.1
4000
Frequency (Hz)
8000
250
125
125 0
4000
ε = ( 40%, 25%) αscorr =
αcorr = t
[ [
6% 23% 6% 20%
] ]
10% 12% 5% 7%
235/16a03.a1e.1
0
50
100 150 Time (ms)
200
250
Very different, Both quadrant separable (somewhat noisy though) ε = ( 60%, 60%) αscorr =
αcorr = t
[ [
32% 40% 21% 27%
] ]
55% 25% 32% 59%
Institute for Systems Research University of Maryland
Selected References Spectro-Temporal Correlation Methods ❑ Klein DJ, Depireux DA, Simon JZ and Shamma SA, Robust spectro-temporal reverse correlation for the auditory system: Optimizing stimulus design, J. Computational Neurosci. 2000. ❑ Eggermont JJ, Hearing Research 66 (1993) 177-201. Singular Value Decomposition ❑ Hansen PC, Rank-Deficient and Discrete Ill-Posed Problems, SIAM 1998. ❑ Press WH, Flannery BP, Teukolsky SA, and Vetterling, WT, Numerical Recipes, Cambridge University Press 1986. Separability ❑ Watson AB and Ahumada AJ, J. Opt. Soc. Am. A2(2) (1985) 322–341. ❑ Saul AB and Humphrey AL, J. Neurophysiol. 64 (1990) 206-224. Related techniques and models ❑ Kowalski N, Depireux D and Shamma S, J. Neurophysiol. 76 (5) (1996) 3503-3523, & 3524-3534. ❑ Depireux DA, Simon JZ and Shamma SA, Comments in Theoretical Biology (1998). ❑ Wang K and Shamma SA, IEEE Trans. on Speech and Audio 2(3) (1994) 421-435, and 3(2) (1995) 382-395.
Special thanks to Steve Bierer for his help in spike sorting.
Center for Auditory and Acoustic Research
Institute for Systems Research University of Maryland