universal neuron - Inst.eecs.berkeley.edu
Tutorials and Reviews International Journal of Bifurcation and Chaos, Vol. 13, No. 9 (2003) 2377–2491 c World Scientific Publishing Company
A NONLINEAR DYNAMICS PERSPECTIVE OF WOLFRAM’S NEW KIND OF SCIENCE. PART II: UNIVERSAL NEURON LEON O. CHUA, VALERY I. SBITNEV and SOOK YOON Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, CA, 94720, USA Received March 1, 2003; Revised May 15, 2003 Wolfram’s celebrated three-input Cellular Automata is further developed and extended from the perspective of neural networks. A single explicit formula involving two nested absolute-value functions and eight adjustable parameters called synaptic weights, is presented. Such a neuron is proved to be universal by specifying the synaptic weights of all 256 local rules. Applying the nonlinear dynamics concepts developed from Part I of this multipart series of papers, we present the rational for partitioning the entire set of 256 local rules into 16 distinct gene families, each composed of 16 gene siblings. Such a partitioning allows us to explain, if not predict, the pattern features generated from each local rule. Finally, these 16 gene families of Cellular Automata rules are encoded onto a new compact and insightful representation called the “double-helix torus.” Keywords: Cellular Automata; cellular neural networks; CNN; universal neuron; universal Turing machine; double-helix torus.
1. Firing and Quenching Patterns The 256 local rules studied incisively by Wolfram in A New Kind of Science [Wolfram, 2002] for the one-dimensional binary Cellular Automata (1D CA) shown in Fig. 1 are encoded onto the vertices of 256 corresponding Boolean cubes and listed in Table 1 of [Chua et al., 2002]. This table is redrawn in the following four pages (Table 1) to emphasize the correlation between the red vertices of each Boolean cube with the corresponding firing patterns of an artificial neuron whose “sphere of influence” consists of its two nearest neighbors [Chua, 1998]. Each local rule “N ” of a 1D CA can be decoded from the corresponding Boolean cube “N ” in Table 1 by associating each RED vertex of this cube with the logic state 1, and each BLUE vertex with the logic state −1 (or logic state 0 when {0, 1} are chosen as the binary states).
It is important to remember that, for reasons already articulated in Part I, we will continue to use {−1, 1} as the logic states in this paper, unless specified otherwise. The only exception is when one wishes to translate the last column (γ7 γ6 γ5 γ4 γ3 γ2 γ1 γ0 ) of the truth table (lower-right corner in Table 1) into a binary format (β7 β6 β5 β4 β3 β2 β1 β0 ), in which case we simply change each γj = −1 to βj = 0. Observe that βj and γj are related via the one-to-one mapping γj = 2βj − 1, or equivalently, βj = 12 (γj + 1). The number N below each Boolean cube in Table 1 is the decimal expression of the corresponding binary number (β7 β6 β5 β4 β3 β2 β1 β0 ); namely,
2377
N = β7 • 2 7 + β 6 • 2 6 + β 5 • 2 5 + β 4 • 2 4 + β3 • 2 3 + β2 • 2 2 + β1 • 2 1 + β0 • 2 0 =
7 k=0
βk 2 k .
(1)
2378
L. O. Chua et al.
Cell (N-2) Cell Cell (N-1) N
Cell 0
Cell (i-1)
Cell Cell 1 2
Cell i
Cell (i+1)
(a)
input ui −1 ui ui +1
state xi
output yi
(b) Fig. 1. (a) A one-dimensional Cellular Automata (CA) made of (N +1) identical cells with a periodic boundary condition. Each cell “i” is coupled only to its left neighbor cell (i–1) and right neighbor cell (i+1). (b) Each cell “i” has a state variable xi (t), an output variable yi (t), and three constant binary inputs ui−1 , ui and ui+1 .
Observe that each vertex k of the Boolean cube is associated with a corresponding multiplication factor 2k (depicted in the lower-left corner in Table 1). Consequently, we can calculate the decimal number N associated with each Boolean cube by simply adding the multiplication factors associated with all RED vertices. For example, the Boolean cube 110 in Table 1 has five RED vertices (6, 5, 3, 2, 1), whose corresponding multiplication factors are 26 = 64, 25 = 32, 23 = 8, 22 = 4, and 21 = 2. Hence, N = 64 + 32 + 8 + 4 + 1 = 110 as expected.
Recall from [Chua et al., 2002] that the Boolean cube ID number N is printed in Red in Table 1 if its associated local rule N is linearly separable. Otherwise, N is printed in Blue. Observe that only 104 out of 256 local rules are linearly separable. It is very important to remember that the decimal number N in Table 1 plays two roles: As an identification (ID) number in decimal format, or as a code for specifying the truth table (local rule) when N is translated into its binary format [Chua, 1998; Wolfram, 2002].
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II Table 1. 2
Encoding 256 local rules defining a binary 1D CA onto 256 corresponding “Boolean Cubes”.
3
2
7
6 4
1 4
4
5
8 3 1 4
16 3
2
7 1
0 4
5
4
3 1 4
3 7 1 4
3 6 4
3
2
7
6
1
4
5
4
3
2
7
6 1
1
4
5
4
59
N = decimal equivalent of binary number β
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
1 5
4
5
47 3
2 6
5
5
1 5
4
2
5
4
5
62
63
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
u it+1
Rule N
1
0
u it−1
23 = 8
3 7
6 1
0
61
55 3
7
6
1
0 4
54
7
3 7
1
2
1
0 4
k
0
21 = 2
4
3
0
60 vertex
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
2 6
0 4
5
3 7
1
0
53 3
7
6
5
58
4
2
1
0
5
52 3
7
6
0
57
4
2 6
7
6 1
0
39
5
2
7
6
5
46 3
1
0 4
3
0 4
3 7
1
7
6
5
2
1
0
51
2
7
6
5
50
5
56
4
3
2 6
5
2
1
0
5
31
38 3
1
0
3
0
3 7
4
7
4
45
2
1
1
7
4
44 3
0
1 5
2 6
1
2
37 6
23
5
6
5
2
5
30 3
0
3
0 4
7
6
5
3
0
2
1
0
7
6
0 4
6
49
2
3 7
1 5
48
5
4
1
0 4
3
0
7
4
7
6
43
2
1 5
2
1
0 4
42 3
0
5
1 5
2
36 3
7
6
0 4
7
1
0 4
1
2
7
2
1
3 7
1
7
6
5
6
0 4
35
7
6
1
3
2 6
5
2
29
7
15
22
7
4
2 6
5
3
5
5
3
0 4
3
0
28 3
0 4
2
41
2
1 5
1
0 4
7
34
5
40
2 6
0
3
0
5
3
4
7
6
0
6
1
2
5
1
1
0 4
7
6
21 6
5
3 7
6 1
2
5
2
7 2
14 3
0
3 7
6
4
7
4
2
27
7
33
2
4
2 6
5
32
4
3
0
5
6
5
1 5
20 1
0
26
7
6
0 4
4
2
7
1
2 6
0
3 7
6
0
25
2 6
2
7
1 5
24
3
7
4
19
2 6
0
5
3
5
3
0
5
1
0 4
7
6
13
2
1
0 4
18 3
7
6
1 5
4
5
3 7
1
2
1
0
2 6
6 3
7
6
5
6
4
5 1
3 7
0
5
2
12 3
7
6
0 4
17
2 6
1 5
2
7
6
0
5
3
2 6
1
0
3 7
4
3 7
4
0
11
2
7
6
0 4
3
5
6
5
10
2
7
6
4
1
2
1
0
5
9
2
3 7
6 1
4
4
2 6
4
2
7 0
3 7
0
3 3
6 1
0
5
5
2 2
7
6 1
4
2 6
1
0
5
3
3 7
1
4
2
7 0
2 6
0
1 3
3 7
1 5
0 2
4
2 6
0
5
6
3 7
6
0
2379
u it +1
2 3
2380
L. O. Chua et al. Table 1. 2
3
2
7
6
1
0 4
4
3 7
6
5
4
3 1 4
3
2
7
4
7
6 1
0 5
4
3
2
7 1
0 4
4
96 3
2
7
6 4
4
104 3 1 4
112
2
1 4
3
2
7 1
0 5
4
1
0
5
4
4
1 5
122
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
125
5
119
7
2
4
126
5
127
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
111
118 3
7
6
vertex
21 = 2
2
124
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
117
1
0 4
7
4
3 7
6
3
0
5
3
0 4
123
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
103
110 3
7
1 5
6
0
5
121
2
116 3
7
6
109 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
2
7
1
1
115 3
6
0
120
2
2 6
5
114 3
7
6
7
4
4
108 3
0
5
5
2 6
5
2
1
0
95
102 3
7
6 1
0 4
2
1
0
5
101 2
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3
2
1
2
1
0 4
7
6
107 6
1 5
87
94 3
7
6
2
1 5
3 7
6
113
2
4
106
0
5
4
5
3 7
6
0
6
4
2
7
4
5
2
100
0
4
93
0
3 7
6 1
5
3 7
4
2
7 0
105
2
3
6 1
0
5
6
7
6 1
0
1 5
99
2
4
2 6
0 4
98 3
3 7
1 5
5
5
3
0
1
0 4
7
6
3 7
1 5
2
1
0
2 6
86 3
7
6
92
2 6
0 4
97
2
3 7
1 5
4
2
1
0
91
2 6
0
5
5
90 3
7
6
4
7
6 1
3
5
79
7
4
1
0 4
3
0
5
3 7
6
5
6
85
2
2
1
2
1
0 4
84
0
5
89
2 6
4
1 5
71
78 3
7
6
0
3 7
6 1
7
4
2
7 0
5
88
3
6 1
0
1 5
4
77 2
5
3
0
1
0 4
7
6
5
3
5
3 7
1
2
1
0 4
2
83
2
5
6
0 4
82 3
3 7
1 5
6 1
2 6
70 3
7
76
2 6
0 4
81
2 6
1 5
80
3 7
6
0
5
75
2
7
6
0 4
3
4
69 2
3 7
0
5
3
0 4
2 6
1
0 4
7
6
5
74
2
7
6
4
5
3 7
1
2
1
0
5
73
2
6
2 6
68 3
7
1
0 4
4
2
7
6
5
72
3
3 7
0
67
2
1
0
1 5
66
7
1
0
4
2 6
0
5
3
3 7
1
4
2
2 6
0
65
2
3 7
1 5
64
4
2 6
0
5
6
3 7
6
(Continued )
u it +1
2 3
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II Table 1. 2
3
2
7
6
1
0 4
4
3 7
6 4
136 3 1 4
144 3
2
7
4
152 3
2
7 1
4
161
2
3
2
7
4
168 3
3
1 4
176
1 4
3
2
7
7
6 1
0
1
0
5
4
184
1
0
5
4
4
1 5
186
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
189
5
183
7
2
4
190
5
191
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
175
182 3
7
6
vertex
21 = 2
2
188
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
181
1
0 4
7
4
3 7
6
3
0
5
3
0 4
187
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
167
174 3
7
1 5
6
0
5
185
3 7
6
2
180
2
7
173 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
179 3
6
1 5
178
4
2 6
0 4
2
3 7
1
5
2 6
5
2
1
0
5
159
166 3
7
6
172
2
5
3
4
2
1
0
171 6
0
5
177
2
5
3 7
6
0
5
4
2
7
6
0
4
5
165
1
0 4
7
4
3 7
6
3
0
5
3 7
6
2
1
0 4
2
1 5
6
5
151
158 3
7
1 5
2
1
0
170
2
7
6
4
169
2
4
5
3 7
6
2
164
2
1
157 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
163
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
162 3
3 7
1 5
5
2 6
5
2
1
0
143
150 3
7
6
156
2 6
0 4
4
2
1
0
155 3
7
1 5
160
4
2
7
6
5
154 6
0
5
4
149
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3
2
1
2
1
0 4
135
142 3
7
1 5
2
1
0
5
3 7
6
0
6
4
153
2
4
5
3 7
6
2
148
2
1
141 6
0
4
5
3
0
1
0 4
7
6
5
3 7
4
147
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
146 3
3 7
1 5
5
5
3 7
1
2
1
0
140
2 6
0 4
145
2
4
1 5
4
139 3
7
6
0
5
2
6
2 6
134 3
7
1
0
4
133 2
3 7
0
5
3 7
6
2 6
1
0 4
2
1 5
138
5
3 7
1
132
0 4
2 6
0
3 7
6
3 7
4
2
5
3 7
6
0
6
4
2
7
4
5
1
131 1
0
137
2 6
6
2 6
5
3 7
1
0
5
2
7
1
0
4
130 3
3
0
5
(Continued )
7
1
4
2
2 6
0
129
2
3 7
1 5
128
4
2 6
0
5
6
3 7
6
2381
u it +1
2 3
2382
L. O. Chua et al. Table 1. 2
3
2
7
6
1
0 4
4
3 7
6 4
200 3 1 4
208 3
2
7
4
216 3
2
7 1
0
4
225
2
3
2
7
4
4
232 3
3
1 4
240
1 4
3
2
7
7
6 1
0
1
0
5
4
248
1
0
5
4
4
1 5
250
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
253
5
247
7
2
4
254
5
255
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
239
246 3
7
6
vertex
21 = 2
2
252
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
245
1
0 4
7
4
3 7
6
3
0
5
3
0 4
251
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
231
238 3
7
1 5
6
0
5
249
3 7
6
2
244
2
7
237 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
243 3
6
1 5
242
4
2 6
0 4
2
3 7
1
5
2 6
5
2
1
0
5
223
230 3
7
6
236
2
5
3
4
2
1
0
235 6
0
5
241
2
5
3 7
6
0
5
4
2
7
6
0
4
5
229
1
0 4
7
4
3 7
6
3
0
5
3 7
6
2
1
0 4
2
1 5
6
5
215
222 3
7
1 5
2
1
0
234
2
7
6
4
233
2
4
5
3 7
6
2
228
2
1
221 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
227
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
226 3
3 7
1 5
5
2 6
5
2
1
0
207
214 3
7
6
220
2 6
0 4
4
2
1
0
219 3
7
1 5
224 6
2
213
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3 7
6
5
218 6
0
5
4
2
1
0
5
3 7
6
7
6 1
4
217
2
4
5
3
2
1
2
1
0 4
199
206 3
7
1 5
212
2
7 0
2 6
0
211
205
7
4
4
5
3
0
1
0 4
7
6
5
3
5
3 7
1
2
1
0 4
2
1 5
3
6 1
0
5
6
7
6 1
0
2
5
6
0 4
210 3
3 7
1 5
6
2 6
198 3
7
1
204
2 6
0 4
209
2
4
1 5
203 3
7
6
0
5
2
4
4
197 2
3 7
0
5
3
0
2 6
1
0 4
7
6
5
202
5
3 7
1
2
1
0 4
2 6
196 3
7
6
5
3 7
6
0
6
4
2
7
4
5
4
2
1
0
201
2 6
6
3 7
0
195 3
7
1
0
5
2
1 5
194
7
1
0
4
2 6
0
5
3
3 7
1
4
2
2 6
0
193
2
3 7
1 5
192
4
2 6
0
5
6
3 7
6
(Continued )
u it +1
2 3
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II
Finally, observe that a three-pixel colored sequence, henceforth called a neighborhood pattern Pk , is printed next to each vertex k in the lower-left corner of Table 1. The color sequence of each neighborhood pattern Pk is defined by the coordinates (uti−1 , uti , uti+1 ) of vertex k in the truth table (bottom-right of Table 1), where “1” is encoded in red, and “−1” is encoded in blue. For example, the neighborhood pattern P3 corresponding to vertex 3 of every Boolean cube in Table 1 has the same color sequence (blue, red, red). A neighborhood pattern Pk is said to be a firing pattern for a local rule N iff the vertex k of the corresponding Boolean cube N in Table 1 is colored in RED. Otherwise, it is called a quenching pattern. The significance of the firing patterns is that a cell Ci in any one-dimensional input pattern (see Fig. 1) will be RED in the next generation if, and only if, the color pattern of the three contiguous cells (Ci−1 , Ci , Ci+1 ) coincides with one of the firing patterns of the Boolean cube. For example, for the local rule 110, only the five firing patterns shown in Table 2 will result in a red cell at Ci in the next generation. In neural network parlance, a neuron Ci
2383
emulating local rule 110 will “fire” (by switching to 1 in a CNN [Chua, 1998], or by initiating an action potential in a real neuron), if, and only if, it recognizes one of the five firing patterns in Table 2. Table 3 shows the quenching patterns associated with the local rule 110. Note that they correspond to the blue vertices in the Boolean cube 110 in Table 1. Just like the synapses of a neuron can be either excitatory, or inhibitory, the neighborhood patterns associated with a local rule can be either “firing” or “quenching”. It is the subtle combination of the “firing” and “quenching” patterns of a local rule which gives rise to its information processing capabilities [Chua & Roska, 2002].
2. A Universal Neuron The main result from [Chua et al., 2002] is a theorem asserting that each of the 256 local rules studied in [Wolfram, 2002] for the 1D CA shown in Fig. 1 can be generated from a single scalar nonlinear differential equation with at most eight parameters. This 8-parameter family of differential equations is given explicitly by:
x˙ i = − xi + xi + 1 − xi − 1 + z2 + c2 z1 + c1 (z0 + b1 ui−1 + b2 ui + b3 ui+1) xi(0) = 0 i = 0, 1, 2, . . . , N .1
(2)
For each local rule N = 0, 1, 2, . . . , 255 from Table 1, there exists at least one parameter vector [c2 , c1 , z2 , z1 , z0 , b1 , b2 , b3 ] ∈ R8 such that Eq. (2) converges to an equilibrium point xi (Q) for each cell Ci . Moreover, the output ∆ 1 (3) yi (t) = (|xi (t) + 1| − |xi (t) − 1|) 2 of cell Ci in Fig. 1 converges to either yi (Q) = 1 or yi (Q) = −1 as prescribed by the Boolean cube N for each of the eight distinct inputs (ui−1 , ui , ui+1 ) from the truth table (bottom-right corner of Table 1). In particular,
yi (Q) = sgn z2 + c2 z1 + c1 (z0 + b1ui−1 + b2 ui + b3 ui+1)
(4)
1 We have abused our notation by using N to denote either the number of cells in Fig. 1, or the ID number of a local rule. The choice will be clear from the context.
2384
L. O. Chua et al. Table 2.
Vertex
Firing Patterns for local rule 110.
number
Decimal code
Equivalent Boolean code
1
(-1,-1, 1)
(0,0,1)
P1
2
(-1, 1,-1)
(0,1,0)
P2
3
(-1,1,1)
(0,1,1)
P3
5
(1,-1,1)
(1,0,1)
P5
6
(1,1,-1)
(1,1,0)
P6
Table 3.
Vertex
Firing pattern Pk
Quenching Patterns for local rule 110.
number
Decimal code
Equivalent Boolean code
0
(-1,-1,-1)
(0,0,0)
P0
4
(1,-1,-1)
(1,0,0)
P4
7
(1,1,1)
(1,1,1)
P7
where2 ∆
sgn[x] =
1, −1 ,
Quenching pattern Pk
x>0 x
A NONLINEAR DYNAMICS PERSPECTIVE OF WOLFRAM’S NEW KIND OF SCIENCE. PART II: UNIVERSAL NEURON LEON O. CHUA, VALERY I. SBITNEV and SOOK YOON Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, CA, 94720, USA Received March 1, 2003; Revised May 15, 2003 Wolfram’s celebrated three-input Cellular Automata is further developed and extended from the perspective of neural networks. A single explicit formula involving two nested absolute-value functions and eight adjustable parameters called synaptic weights, is presented. Such a neuron is proved to be universal by specifying the synaptic weights of all 256 local rules. Applying the nonlinear dynamics concepts developed from Part I of this multipart series of papers, we present the rational for partitioning the entire set of 256 local rules into 16 distinct gene families, each composed of 16 gene siblings. Such a partitioning allows us to explain, if not predict, the pattern features generated from each local rule. Finally, these 16 gene families of Cellular Automata rules are encoded onto a new compact and insightful representation called the “double-helix torus.” Keywords: Cellular Automata; cellular neural networks; CNN; universal neuron; universal Turing machine; double-helix torus.
1. Firing and Quenching Patterns The 256 local rules studied incisively by Wolfram in A New Kind of Science [Wolfram, 2002] for the one-dimensional binary Cellular Automata (1D CA) shown in Fig. 1 are encoded onto the vertices of 256 corresponding Boolean cubes and listed in Table 1 of [Chua et al., 2002]. This table is redrawn in the following four pages (Table 1) to emphasize the correlation between the red vertices of each Boolean cube with the corresponding firing patterns of an artificial neuron whose “sphere of influence” consists of its two nearest neighbors [Chua, 1998]. Each local rule “N ” of a 1D CA can be decoded from the corresponding Boolean cube “N ” in Table 1 by associating each RED vertex of this cube with the logic state 1, and each BLUE vertex with the logic state −1 (or logic state 0 when {0, 1} are chosen as the binary states).
It is important to remember that, for reasons already articulated in Part I, we will continue to use {−1, 1} as the logic states in this paper, unless specified otherwise. The only exception is when one wishes to translate the last column (γ7 γ6 γ5 γ4 γ3 γ2 γ1 γ0 ) of the truth table (lower-right corner in Table 1) into a binary format (β7 β6 β5 β4 β3 β2 β1 β0 ), in which case we simply change each γj = −1 to βj = 0. Observe that βj and γj are related via the one-to-one mapping γj = 2βj − 1, or equivalently, βj = 12 (γj + 1). The number N below each Boolean cube in Table 1 is the decimal expression of the corresponding binary number (β7 β6 β5 β4 β3 β2 β1 β0 ); namely,
2377
N = β7 • 2 7 + β 6 • 2 6 + β 5 • 2 5 + β 4 • 2 4 + β3 • 2 3 + β2 • 2 2 + β1 • 2 1 + β0 • 2 0 =
7 k=0
βk 2 k .
(1)
2378
L. O. Chua et al.
Cell (N-2) Cell Cell (N-1) N
Cell 0
Cell (i-1)
Cell Cell 1 2
Cell i
Cell (i+1)
(a)
input ui −1 ui ui +1
state xi
output yi
(b) Fig. 1. (a) A one-dimensional Cellular Automata (CA) made of (N +1) identical cells with a periodic boundary condition. Each cell “i” is coupled only to its left neighbor cell (i–1) and right neighbor cell (i+1). (b) Each cell “i” has a state variable xi (t), an output variable yi (t), and three constant binary inputs ui−1 , ui and ui+1 .
Observe that each vertex k of the Boolean cube is associated with a corresponding multiplication factor 2k (depicted in the lower-left corner in Table 1). Consequently, we can calculate the decimal number N associated with each Boolean cube by simply adding the multiplication factors associated with all RED vertices. For example, the Boolean cube 110 in Table 1 has five RED vertices (6, 5, 3, 2, 1), whose corresponding multiplication factors are 26 = 64, 25 = 32, 23 = 8, 22 = 4, and 21 = 2. Hence, N = 64 + 32 + 8 + 4 + 1 = 110 as expected.
Recall from [Chua et al., 2002] that the Boolean cube ID number N is printed in Red in Table 1 if its associated local rule N is linearly separable. Otherwise, N is printed in Blue. Observe that only 104 out of 256 local rules are linearly separable. It is very important to remember that the decimal number N in Table 1 plays two roles: As an identification (ID) number in decimal format, or as a code for specifying the truth table (local rule) when N is translated into its binary format [Chua, 1998; Wolfram, 2002].
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II Table 1. 2
Encoding 256 local rules defining a binary 1D CA onto 256 corresponding “Boolean Cubes”.
3
2
7
6 4
1 4
4
5
8 3 1 4
16 3
2
7 1
0 4
5
4
3 1 4
3 7 1 4
3 6 4
3
2
7
6
1
4
5
4
3
2
7
6 1
1
4
5
4
59
N = decimal equivalent of binary number β
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
1 5
4
5
47 3
2 6
5
5
1 5
4
2
5
4
5
62
63
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
u it+1
Rule N
1
0
u it−1
23 = 8
3 7
6 1
0
61
55 3
7
6
1
0 4
54
7
3 7
1
2
1
0 4
k
0
21 = 2
4
3
0
60 vertex
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
2 6
0 4
5
3 7
1
0
53 3
7
6
5
58
4
2
1
0
5
52 3
7
6
0
57
4
2 6
7
6 1
0
39
5
2
7
6
5
46 3
1
0 4
3
0 4
3 7
1
7
6
5
2
1
0
51
2
7
6
5
50
5
56
4
3
2 6
5
2
1
0
5
31
38 3
1
0
3
0
3 7
4
7
4
45
2
1
1
7
4
44 3
0
1 5
2 6
1
2
37 6
23
5
6
5
2
5
30 3
0
3
0 4
7
6
5
3
0
2
1
0
7
6
0 4
6
49
2
3 7
1 5
48
5
4
1
0 4
3
0
7
4
7
6
43
2
1 5
2
1
0 4
42 3
0
5
1 5
2
36 3
7
6
0 4
7
1
0 4
1
2
7
2
1
3 7
1
7
6
5
6
0 4
35
7
6
1
3
2 6
5
2
29
7
15
22
7
4
2 6
5
3
5
5
3
0 4
3
0
28 3
0 4
2
41
2
1 5
1
0 4
7
34
5
40
2 6
0
3
0
5
3
4
7
6
0
6
1
2
5
1
1
0 4
7
6
21 6
5
3 7
6 1
2
5
2
7 2
14 3
0
3 7
6
4
7
4
2
27
7
33
2
4
2 6
5
32
4
3
0
5
6
5
1 5
20 1
0
26
7
6
0 4
4
2
7
1
2 6
0
3 7
6
0
25
2 6
2
7
1 5
24
3
7
4
19
2 6
0
5
3
5
3
0
5
1
0 4
7
6
13
2
1
0 4
18 3
7
6
1 5
4
5
3 7
1
2
1
0
2 6
6 3
7
6
5
6
4
5 1
3 7
0
5
2
12 3
7
6
0 4
17
2 6
1 5
2
7
6
0
5
3
2 6
1
0
3 7
4
3 7
4
0
11
2
7
6
0 4
3
5
6
5
10
2
7
6
4
1
2
1
0
5
9
2
3 7
6 1
4
4
2 6
4
2
7 0
3 7
0
3 3
6 1
0
5
5
2 2
7
6 1
4
2 6
1
0
5
3
3 7
1
4
2
7 0
2 6
0
1 3
3 7
1 5
0 2
4
2 6
0
5
6
3 7
6
0
2379
u it +1
2 3
2380
L. O. Chua et al. Table 1. 2
3
2
7
6
1
0 4
4
3 7
6
5
4
3 1 4
3
2
7
4
7
6 1
0 5
4
3
2
7 1
0 4
4
96 3
2
7
6 4
4
104 3 1 4
112
2
1 4
3
2
7 1
0 5
4
1
0
5
4
4
1 5
122
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
125
5
119
7
2
4
126
5
127
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
111
118 3
7
6
vertex
21 = 2
2
124
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
117
1
0 4
7
4
3 7
6
3
0
5
3
0 4
123
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
103
110 3
7
1 5
6
0
5
121
2
116 3
7
6
109 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
2
7
1
1
115 3
6
0
120
2
2 6
5
114 3
7
6
7
4
4
108 3
0
5
5
2 6
5
2
1
0
95
102 3
7
6 1
0 4
2
1
0
5
101 2
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3
2
1
2
1
0 4
7
6
107 6
1 5
87
94 3
7
6
2
1 5
3 7
6
113
2
4
106
0
5
4
5
3 7
6
0
6
4
2
7
4
5
2
100
0
4
93
0
3 7
6 1
5
3 7
4
2
7 0
105
2
3
6 1
0
5
6
7
6 1
0
1 5
99
2
4
2 6
0 4
98 3
3 7
1 5
5
5
3
0
1
0 4
7
6
3 7
1 5
2
1
0
2 6
86 3
7
6
92
2 6
0 4
97
2
3 7
1 5
4
2
1
0
91
2 6
0
5
5
90 3
7
6
4
7
6 1
3
5
79
7
4
1
0 4
3
0
5
3 7
6
5
6
85
2
2
1
2
1
0 4
84
0
5
89
2 6
4
1 5
71
78 3
7
6
0
3 7
6 1
7
4
2
7 0
5
88
3
6 1
0
1 5
4
77 2
5
3
0
1
0 4
7
6
5
3
5
3 7
1
2
1
0 4
2
83
2
5
6
0 4
82 3
3 7
1 5
6 1
2 6
70 3
7
76
2 6
0 4
81
2 6
1 5
80
3 7
6
0
5
75
2
7
6
0 4
3
4
69 2
3 7
0
5
3
0 4
2 6
1
0 4
7
6
5
74
2
7
6
4
5
3 7
1
2
1
0
5
73
2
6
2 6
68 3
7
1
0 4
4
2
7
6
5
72
3
3 7
0
67
2
1
0
1 5
66
7
1
0
4
2 6
0
5
3
3 7
1
4
2
2 6
0
65
2
3 7
1 5
64
4
2 6
0
5
6
3 7
6
(Continued )
u it +1
2 3
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II Table 1. 2
3
2
7
6
1
0 4
4
3 7
6 4
136 3 1 4
144 3
2
7
4
152 3
2
7 1
4
161
2
3
2
7
4
168 3
3
1 4
176
1 4
3
2
7
7
6 1
0
1
0
5
4
184
1
0
5
4
4
1 5
186
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
189
5
183
7
2
4
190
5
191
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
175
182 3
7
6
vertex
21 = 2
2
188
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
181
1
0 4
7
4
3 7
6
3
0
5
3
0 4
187
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
167
174 3
7
1 5
6
0
5
185
3 7
6
2
180
2
7
173 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
179 3
6
1 5
178
4
2 6
0 4
2
3 7
1
5
2 6
5
2
1
0
5
159
166 3
7
6
172
2
5
3
4
2
1
0
171 6
0
5
177
2
5
3 7
6
0
5
4
2
7
6
0
4
5
165
1
0 4
7
4
3 7
6
3
0
5
3 7
6
2
1
0 4
2
1 5
6
5
151
158 3
7
1 5
2
1
0
170
2
7
6
4
169
2
4
5
3 7
6
2
164
2
1
157 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
163
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
162 3
3 7
1 5
5
2 6
5
2
1
0
143
150 3
7
6
156
2 6
0 4
4
2
1
0
155 3
7
1 5
160
4
2
7
6
5
154 6
0
5
4
149
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3
2
1
2
1
0 4
135
142 3
7
1 5
2
1
0
5
3 7
6
0
6
4
153
2
4
5
3 7
6
2
148
2
1
141 6
0
4
5
3
0
1
0 4
7
6
5
3 7
4
147
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
146 3
3 7
1 5
5
5
3 7
1
2
1
0
140
2 6
0 4
145
2
4
1 5
4
139 3
7
6
0
5
2
6
2 6
134 3
7
1
0
4
133 2
3 7
0
5
3 7
6
2 6
1
0 4
2
1 5
138
5
3 7
1
132
0 4
2 6
0
3 7
6
3 7
4
2
5
3 7
6
0
6
4
2
7
4
5
1
131 1
0
137
2 6
6
2 6
5
3 7
1
0
5
2
7
1
0
4
130 3
3
0
5
(Continued )
7
1
4
2
2 6
0
129
2
3 7
1 5
128
4
2 6
0
5
6
3 7
6
2381
u it +1
2 3
2382
L. O. Chua et al. Table 1. 2
3
2
7
6
1
0 4
4
3 7
6 4
200 3 1 4
208 3
2
7
4
216 3
2
7 1
0
4
225
2
3
2
7
4
4
232 3
3
1 4
240
1 4
3
2
7
7
6 1
0
1
0
5
4
248
1
0
5
4
4
1 5
250
7
β
6
β
β
5
β
4
(-1,1,-1) 2 22 = 4
26 = 64
6 20 = 1
k
4
u it−1
3
2
(-1,1,1)
(1,1,1) 7 0 (-1,-1,-1)
(1,-1,-1)
β
5
3
27 = 128 1 (-1,-1,1) 25 = 32
t +1 t t t u i ( i −1 i i +1 t +1 t t t u i ( i −1 i i +1
u
u
1
β
,u ,u ) = 1 ,u ,u ) = −1
7 1 5
4
5
4
253
5
247
7
2
4
254
5
255
u it−1
u it
u it+1
0
-1
-1
-1
γ
0
1
-1
-1
1
γ
1
2
-1
1
-1
3
-1
1
1
γ γ
4
1
-1
-1
γ
4
5
1
-1
1
γ
5
βk = 1
6
1
1
-1
γ
6
βk = 0
7
1
1
1
γ
7
23 = 8
u
t i +1
Rule N
1
0
k
0
3 7
6 1
5
1
0 4
3
0
3 7
1
2
1
0
2 6
5
6
5
239
246 3
7
6
vertex
21 = 2
2
252
u it
(1,-1,1)
24 = 16
vertex k
(1,1,-1)
β
245
1
0 4
7
4
3 7
6
3
0
5
3
0 4
251
N = decimal equivalent of binary number β
2
2
1
0 4
2
1 5
6
5
231
238 3
7
1 5
6
0
5
249
3 7
6
2
244
2
7
237 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
243 3
6
1 5
242
4
2 6
0 4
2
3 7
1
5
2 6
5
2
1
0
5
223
230 3
7
6
236
2
5
3
4
2
1
0
235 6
0
5
241
2
5
3 7
6
0
5
4
2
7
6
0
4
5
229
1
0 4
7
4
3 7
6
3
0
5
3 7
6
2
1
0 4
2
1 5
6
5
215
222 3
7
1 5
2
1
0
234
2
7
6
4
233
2
4
5
3 7
6
2
228
2
1
221 6
0
4
1
0 4
3
0
3 7
1
7
6
5
3 7
4
227
7 0
1 5
3
6 1
0
5
6
7
6 1
0
2
4
2 6
0 4
226 3
3 7
1 5
5
2 6
5
2
1
0
207
214 3
7
6
220
2 6
0 4
4
2
1
0
219 3
7
1 5
224 6
2
213
5
3
0
1
0 4
7
4
3 7
6
5
6
5
3 7
6
5
218 6
0
5
4
2
1
0
5
3 7
6
7
6 1
4
217
2
4
5
3
2
1
2
1
0 4
199
206 3
7
1 5
212
2
7 0
2 6
0
211
205
7
4
4
5
3
0
1
0 4
7
6
5
3
5
3 7
1
2
1
0 4
2
1 5
3
6 1
0
5
6
7
6 1
0
2
5
6
0 4
210 3
3 7
1 5
6
2 6
198 3
7
1
204
2 6
0 4
209
2
4
1 5
203 3
7
6
0
5
2
4
4
197 2
3 7
0
5
3
0
2 6
1
0 4
7
6
5
202
5
3 7
1
2
1
0 4
2 6
196 3
7
6
5
3 7
6
0
6
4
2
7
4
5
4
2
1
0
201
2 6
6
3 7
0
195 3
7
1
0
5
2
1 5
194
7
1
0
4
2 6
0
5
3
3 7
1
4
2
2 6
0
193
2
3 7
1 5
192
4
2 6
0
5
6
3 7
6
(Continued )
u it +1
2 3
A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part II
Finally, observe that a three-pixel colored sequence, henceforth called a neighborhood pattern Pk , is printed next to each vertex k in the lower-left corner of Table 1. The color sequence of each neighborhood pattern Pk is defined by the coordinates (uti−1 , uti , uti+1 ) of vertex k in the truth table (bottom-right of Table 1), where “1” is encoded in red, and “−1” is encoded in blue. For example, the neighborhood pattern P3 corresponding to vertex 3 of every Boolean cube in Table 1 has the same color sequence (blue, red, red). A neighborhood pattern Pk is said to be a firing pattern for a local rule N iff the vertex k of the corresponding Boolean cube N in Table 1 is colored in RED. Otherwise, it is called a quenching pattern. The significance of the firing patterns is that a cell Ci in any one-dimensional input pattern (see Fig. 1) will be RED in the next generation if, and only if, the color pattern of the three contiguous cells (Ci−1 , Ci , Ci+1 ) coincides with one of the firing patterns of the Boolean cube. For example, for the local rule 110, only the five firing patterns shown in Table 2 will result in a red cell at Ci in the next generation. In neural network parlance, a neuron Ci
2383
emulating local rule 110 will “fire” (by switching to 1 in a CNN [Chua, 1998], or by initiating an action potential in a real neuron), if, and only if, it recognizes one of the five firing patterns in Table 2. Table 3 shows the quenching patterns associated with the local rule 110. Note that they correspond to the blue vertices in the Boolean cube 110 in Table 1. Just like the synapses of a neuron can be either excitatory, or inhibitory, the neighborhood patterns associated with a local rule can be either “firing” or “quenching”. It is the subtle combination of the “firing” and “quenching” patterns of a local rule which gives rise to its information processing capabilities [Chua & Roska, 2002].
2. A Universal Neuron The main result from [Chua et al., 2002] is a theorem asserting that each of the 256 local rules studied in [Wolfram, 2002] for the 1D CA shown in Fig. 1 can be generated from a single scalar nonlinear differential equation with at most eight parameters. This 8-parameter family of differential equations is given explicitly by:
x˙ i = − xi + xi + 1 − xi − 1 + z2 + c2 z1 + c1 (z0 + b1 ui−1 + b2 ui + b3 ui+1) xi(0) = 0 i = 0, 1, 2, . . . , N .1
(2)
For each local rule N = 0, 1, 2, . . . , 255 from Table 1, there exists at least one parameter vector [c2 , c1 , z2 , z1 , z0 , b1 , b2 , b3 ] ∈ R8 such that Eq. (2) converges to an equilibrium point xi (Q) for each cell Ci . Moreover, the output ∆ 1 (3) yi (t) = (|xi (t) + 1| − |xi (t) − 1|) 2 of cell Ci in Fig. 1 converges to either yi (Q) = 1 or yi (Q) = −1 as prescribed by the Boolean cube N for each of the eight distinct inputs (ui−1 , ui , ui+1 ) from the truth table (bottom-right corner of Table 1). In particular,
yi (Q) = sgn z2 + c2 z1 + c1 (z0 + b1ui−1 + b2 ui + b3 ui+1)
(4)
1 We have abused our notation by using N to denote either the number of cells in Fig. 1, or the ID number of a local rule. The choice will be clear from the context.
2384
L. O. Chua et al. Table 2.
Vertex
Firing Patterns for local rule 110.
number
Decimal code
Equivalent Boolean code
1
(-1,-1, 1)
(0,0,1)
P1
2
(-1, 1,-1)
(0,1,0)
P2
3
(-1,1,1)
(0,1,1)
P3
5
(1,-1,1)
(1,0,1)
P5
6
(1,1,-1)
(1,1,0)
P6
Table 3.
Vertex
Firing pattern Pk
Quenching Patterns for local rule 110.
number
Decimal code
Equivalent Boolean code
0
(-1,-1,-1)
(0,0,0)
P0
4
(1,-1,-1)
(1,0,0)
P4
7
(1,1,1)
(1,1,1)
P7
where2 ∆
sgn[x] =
1, −1 ,
Quenching pattern Pk
x>0 x
