A forecast-based biologically-plausible STDP learning rule - APT - The ...
A forecast-based biologically-plausible STDP learning rule Sergio Davies, Alexander Rast, Francesco Galluppi and Steve Furber APT group The University of Manchester
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Overview of topics Standard STDP learning rule Description of the new approach Statistical details involved The STDP-TTS Test environment Learning features IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Spike Timing Dependent Plasticity
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Implementation Triggering the STDP algorithm The usual way:
STDP is triggered on:
The SpiNNaker way:
STDP is triggered only on pre-synaptic spike arrival (LTD and LTP)
Pre-synaptic spike arrival (LTD) Post-synaptic spike emission (LTP)
Weights are available only at presynaptic spike arrival. Since LTP needs future information, the algorithm needs to be deferred until the time window is filled
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
New approach Is it possible to simplify the STDP model so that its implementation on SpiNNaker is more performant (from both memory and computational points of view)? To avoid the Deferred Event Model, we need to have statistics that tell us when a neuron is going to fire in the future (at least with some probability).
Spike Forecast
Current simulation time IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Statistical approach - 1 The idea is: the higher the membrane potential of a neuron (that receives a spike), the sooner it is likely to emit an action potential. Starting with a random network of Izhikevich neurons, fed with input to random neurons with random delays; We store all the activity in the network (especially membrane potential evolution). IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Statistical approach - 2 Raster plot in Matlab (fixed−point)
3500
3000
Neuron ID
2500
2000 1500
1000
500
0
IJCNN 2011 S. Jose, CA, USA
0
100
200
300
400
500 600 Time (ms)
700
800
Sergio Davies A forecast-based biologically-plausible STDP learning rule
900
1000
Statistical approach - 3 Outgoing spikes
We evaluate all the couples (membrane potential; time-to-spike)
Membrane potential Membrane reset potential Firing threshold
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Results of the statistical approach Representation of all the couples computed before Sliding window (512 samples) filtered representation
300
250
200
150
100
50
0 −100
−80
−60
−40
−20
0
20
Membrane potential (mV)
IJCNN 2011 S. Jose, CA, USA
40
Time-To-Spike forecast (msec)
Time-To-Spike forecast (msec)
Raw representation 180
512 160 140 120 100 80 60 40 20 0 −100
−80
−60
−40
−20
0
20
Membrane potential (mV)
Sergio Davies A forecast-based biologically-plausible STDP learning rule
40
The STDP-TTS The wider the STDP time window, the greater the uncertainty of the forecast of the time-to-spike. We limit the STDP time window to 32 msec. Time-To-Spike forecast (msec)
Forecast limited to The learning window 512
30
25
20
15
10
5
0
−80
−60
−40
−20
0
20
Membrane potential (mV)
-65mV
-40mV
“L” mV
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
30mV
Input provided
Neuron ID
Can you identify any pattern in this raster plot?
Simulation time (msec) IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Input provided
Neuron ID
Solution: in red the pattern
Simulation time (msec) IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Testing the features To test this forecast learning rule we use as a benchmark the tests ran by Masquelier et al. in 2008 and 2009.
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Results of the tests - 1 STDP-TTS (with forecast)
Standard STDP
L = -60mV
90 80
Delay after pattern input (msec)
L = -65mV
160 140
70
120
60
100
200 50
180
80 40
160
60 30
140
40
20
120 20
10
100 0
80
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
10 4
4
x 10
x 10
60
L = -70mV
40 140
20 0
L = -90mV
250
120
0
1
2
3
4
5
6
7
8
9
10
200
4
Simulation time (sec)
x 10
100 150 80
60
100
40 50 20
0
0 0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Results of the tests - 2 Two output neurons – one input pattern Standard STDP Neuron 1
200
STDP with forecast Neuron 1
180
180
160
160
140
140
120
120 100 100 80 80 60
60
40
40
20
20 0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
4
Neuron 2
200
x 10
180
160
160
140
140
120
120
100
100
80
80
60
60
40
40
20
20
0
1
2
3
4
5
6
Neuron 2
200
180
0
7
8
9
10
0
0
1
2
3
4
x 10
IJCNN 2011 S. Jose, CA, USA
10 4
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
4
5
6
7
8
9
10 4
x 10
Results of the tests - 3 Four output neurons – two input patterns standard STDP
Pattern 1
Neuron 1
Neuron 2
Neuron 3
300
300
300
300
250
250
250
250
200
200
200
200
150
150
150
150
100
100
100
100
50
50
50
50
0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
4
300
10
0
0
1
2
3
4
5
6
7
8
9
4
x 10
10
0
1
2
3
4
5
6
7
8
9
10 4
x 10
x 10
250
300
300
200
250
250
200
200
150
150
100
100
50
50
200
0
4
x 10
250
Pattern 2
Neuron 4
150 150 100 100
50
50
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
0
0
1
2
3
4
5
6
7
8
9
10 4
0
0
1
2
3
4
5
6
7
8
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
9
10 4
x 10
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Results of the tests - 4 Four output neurons – two input patterns STDP with forecast Neuron 1
Neuron 2
300
Neuron 3
250
250
Neuron 4
300
350
300
250
200
Pattern 1
250 200
200 150
200
150
150 150
100 100
100 100 50
50
0
0
1
2
3
4
5
6
7
8
9
10
0
50
0
1
2
3
4
5
6
7
8
9
10
4
0
1
2
3
4
5
6
7
8
9
4
x 10
300
0
50
10
0
0
1
2
3
4
5
6
7
8
9
4
x 10
10 4
x 10
x 10
250
250
300
200
200
250
150
150
100
100
Pattern 2
250
200 200 150 150 100 100 50
50
50 50
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
0
0
1
2
3
4
5
6
7
8
9
10 4
0
0
1
2
3
4
5
6
7
8
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
9
10 4
x 10
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Thank you!!!
Questions??? IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Backup slides
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Limitations of the new rule Outgoing spikes
STDP with forecast cannot tune to the earliest spikes The forecast function is related to the type of Izhikevich neuron Synaptic plasticity threshold Membrane potential Membrane reset potential IJCNN 2011 Sergio Davies Firing threshold S. Jose, CA, USA A forecast-based biologically-plausible STDP learning rule
The Izhikevich phase space
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Implementation on SpiNNaker LTP 1 LTP 2
LTD 1
LTD 2
Pre
time
Post
time LTP 1 LTD 1
Incoming spike Incoming spike delayed by synapse Synaptic delay
Pre
time
Post
time
Forecast based on the current neuron membrane potential Forecasted spike Real outgoing spike
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Overview of topics Standard STDP learning rule Description of the new approach Statistical details involved The STDP-TTS Test environment Learning features IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Spike Timing Dependent Plasticity
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Implementation Triggering the STDP algorithm The usual way:
STDP is triggered on:
The SpiNNaker way:
STDP is triggered only on pre-synaptic spike arrival (LTD and LTP)
Pre-synaptic spike arrival (LTD) Post-synaptic spike emission (LTP)
Weights are available only at presynaptic spike arrival. Since LTP needs future information, the algorithm needs to be deferred until the time window is filled
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
New approach Is it possible to simplify the STDP model so that its implementation on SpiNNaker is more performant (from both memory and computational points of view)? To avoid the Deferred Event Model, we need to have statistics that tell us when a neuron is going to fire in the future (at least with some probability).
Spike Forecast
Current simulation time IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Statistical approach - 1 The idea is: the higher the membrane potential of a neuron (that receives a spike), the sooner it is likely to emit an action potential. Starting with a random network of Izhikevich neurons, fed with input to random neurons with random delays; We store all the activity in the network (especially membrane potential evolution). IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Statistical approach - 2 Raster plot in Matlab (fixed−point)
3500
3000
Neuron ID
2500
2000 1500
1000
500
0
IJCNN 2011 S. Jose, CA, USA
0
100
200
300
400
500 600 Time (ms)
700
800
Sergio Davies A forecast-based biologically-plausible STDP learning rule
900
1000
Statistical approach - 3 Outgoing spikes
We evaluate all the couples (membrane potential; time-to-spike)
Membrane potential Membrane reset potential Firing threshold
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Results of the statistical approach Representation of all the couples computed before Sliding window (512 samples) filtered representation
300
250
200
150
100
50
0 −100
−80
−60
−40
−20
0
20
Membrane potential (mV)
IJCNN 2011 S. Jose, CA, USA
40
Time-To-Spike forecast (msec)
Time-To-Spike forecast (msec)
Raw representation 180
512 160 140 120 100 80 60 40 20 0 −100
−80
−60
−40
−20
0
20
Membrane potential (mV)
Sergio Davies A forecast-based biologically-plausible STDP learning rule
40
The STDP-TTS The wider the STDP time window, the greater the uncertainty of the forecast of the time-to-spike. We limit the STDP time window to 32 msec. Time-To-Spike forecast (msec)
Forecast limited to The learning window 512
30
25
20
15
10
5
0
−80
−60
−40
−20
0
20
Membrane potential (mV)
-65mV
-40mV
“L” mV
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
30mV
Input provided
Neuron ID
Can you identify any pattern in this raster plot?
Simulation time (msec) IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Input provided
Neuron ID
Solution: in red the pattern
Simulation time (msec) IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Testing the features To test this forecast learning rule we use as a benchmark the tests ran by Masquelier et al. in 2008 and 2009.
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Results of the tests - 1 STDP-TTS (with forecast)
Standard STDP
L = -60mV
90 80
Delay after pattern input (msec)
L = -65mV
160 140
70
120
60
100
200 50
180
80 40
160
60 30
140
40
20
120 20
10
100 0
80
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
10 4
4
x 10
x 10
60
L = -70mV
40 140
20 0
L = -90mV
250
120
0
1
2
3
4
5
6
7
8
9
10
200
4
Simulation time (sec)
x 10
100 150 80
60
100
40 50 20
0
0 0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Results of the tests - 2 Two output neurons – one input pattern Standard STDP Neuron 1
200
STDP with forecast Neuron 1
180
180
160
160
140
140
120
120 100 100 80 80 60
60
40
40
20
20 0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
4
Neuron 2
200
x 10
180
160
160
140
140
120
120
100
100
80
80
60
60
40
40
20
20
0
1
2
3
4
5
6
Neuron 2
200
180
0
7
8
9
10
0
0
1
2
3
4
x 10
IJCNN 2011 S. Jose, CA, USA
10 4
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
4
5
6
7
8
9
10 4
x 10
Results of the tests - 3 Four output neurons – two input patterns standard STDP
Pattern 1
Neuron 1
Neuron 2
Neuron 3
300
300
300
300
250
250
250
250
200
200
200
200
150
150
150
150
100
100
100
100
50
50
50
50
0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
4
300
10
0
0
1
2
3
4
5
6
7
8
9
4
x 10
10
0
1
2
3
4
5
6
7
8
9
10 4
x 10
x 10
250
300
300
200
250
250
200
200
150
150
100
100
50
50
200
0
4
x 10
250
Pattern 2
Neuron 4
150 150 100 100
50
50
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
0
0
1
2
3
4
5
6
7
8
9
10 4
0
0
1
2
3
4
5
6
7
8
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
9
10 4
x 10
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Results of the tests - 4 Four output neurons – two input patterns STDP with forecast Neuron 1
Neuron 2
300
Neuron 3
250
250
Neuron 4
300
350
300
250
200
Pattern 1
250 200
200 150
200
150
150 150
100 100
100 100 50
50
0
0
1
2
3
4
5
6
7
8
9
10
0
50
0
1
2
3
4
5
6
7
8
9
10
4
0
1
2
3
4
5
6
7
8
9
4
x 10
300
0
50
10
0
0
1
2
3
4
5
6
7
8
9
4
x 10
10 4
x 10
x 10
250
250
300
200
200
250
150
150
100
100
Pattern 2
250
200 200 150 150 100 100 50
50
50 50
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
IJCNN 2011 S. Jose, CA, USA
0
0
1
2
3
4
5
6
7
8
9
10 4
0
0
1
2
3
4
5
6
7
8
x 10
Sergio Davies A forecast-based biologically-plausible STDP learning rule
9
10 4
x 10
0
0
1
2
3
4
5
6
7
8
9
10 4
x 10
Thank you!!!
Questions??? IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Backup slides
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Limitations of the new rule Outgoing spikes
STDP with forecast cannot tune to the earliest spikes The forecast function is related to the type of Izhikevich neuron Synaptic plasticity threshold Membrane potential Membrane reset potential IJCNN 2011 Sergio Davies Firing threshold S. Jose, CA, USA A forecast-based biologically-plausible STDP learning rule
The Izhikevich phase space
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
Implementation on SpiNNaker LTP 1 LTP 2
LTD 1
LTD 2
Pre
time
Post
time LTP 1 LTD 1
Incoming spike Incoming spike delayed by synapse Synaptic delay
Pre
time
Post
time
Forecast based on the current neuron membrane potential Forecasted spike Real outgoing spike
IJCNN 2011 S. Jose, CA, USA
Sergio Davies A forecast-based biologically-plausible STDP learning rule
