Finite State Transformations of Images - CiteSeerX
Finite State Transformations of Images Karel Culik II
Department of Computer Science University of South Carolina Columbia, S.C. 29208
Jarkko Kari y
Academy of Finland
Abstract
Weighted nite automata (WFA) have been introduced as devices for computing real functions on [0; 1]n. The main motivation has been to generate functions on [0; 1] [0; 1] interpreted as gray-tone images. Weighted nite transducers (WFT) are nite state devices that serve as a powerful tool for describing and implementing a large variety of image transformations and more generally linear operators on real functions. Here we show new results on WFT and demonstrate that WFT are indeed an excellent tool for image manipulation and more generally for function transformation. We note that every WFA transformation is a linear operator and show that most of the interesting linear operators on real functions (on [0; 1]2) can be easily implemented by WFT. We give a number of examples that include ane transformations, a low-pass lter, wavelet transform, (partial) derivatives, simple and multiple integrals. Since the family of WFA-functions is constructively closed under WFT, each of our examples is actually a proof of a theorem stating that for each WFA A there e ectively exists another WFA B that computes the integral (or other transformations) of the function de ned by A.
Research was supported by the National Science Foundation under Grant No. CCR-9202396. Preliminary version of this paper has been presented at ICALP 95. yThe address of the second author is: Mathematics Department, University of Turku, 20500 Turku, Finland
0
1 Introduction Weighted nite automata (WFA) have been introduced in [3, 4] as devices for computing real functions on [0; 1]n . The main motivation has been to generate functions on [0; 1] [0; 1] interpreted as gray-tone images. In [4, 5] (see also [6]) we developed inference algorithms for WFA. Using the algorithm from [5] we can eciently encode any image (digitalized photograph) by a WFA. One of the best image compression method has been developed on this basis. In [2] the generalized k-tape WFA have been introduced and in particular, the 2tape WFA called weighted nite transducers (WFT) have been studied and shown to perform a number of simple operations on images. When considered as only de ning mappings on nite words WFT are a special case of rational transducers [9]. In [7] the iterative WFT have been studied and shown to be strictly more powerful image generators than the mutual recursive function systems (MRFS), see e.g. [1]. Here we show new results on WFT and demonstrate that WFT are indeed an excellent tool for image manipulation and more generally for function transformation. We note that every WFA transformation is a linear operator and show that most of the interesting linear operators on real functions (on [0; 1]2) can be easily implemented by the WFT. We give a number of examples that include ane transformations, a low-pass lter, wavelet transform, (partial) derivatives, simple and multiple integrals. Since the family of WFA-functions is constructively closed under WFT [2], each of our examples is actually a proof of a theorem stating that for each WFA A there e ectively exists another WFA B that computes the integral (or other transformations) of the function de ned by A. In the case of integrals this has been shown in [3] for average preserving WFA. We extend this result to arbitrary WFA and to multiple integrals. In Section 2 we introduce an automata-theoretic notation for multiresolution images and weighted nite automata and transducers. In Section 3 we introduce average preserving WFT and show that average preserving WFA are closed under average preserving WFT. This is important since an average preserving WFA generates an average preserving function that is a well de ned multiresolution image. We show that average preserving WFT are closed under the operations of composition, addition, and multiplication by a constant. In Section 4 we demonstrate the surprising ability and exibility of WFT to implement image transformations and linear function operators. We show that every piecewise ane transformation can be implemented by WFT. The crucial, somewhat tricky example, of a WFT is the one that shifts an image by one pixel for any nite resolution. It leads us to implementation of lters, (partial) derivatives, and other linear operators.
1
2 Multiresolution functions, WFA and WFT
A multiresolution function f over alphabet is a function ?! IR. It is average preserving if X f (ua) = p f (u); for all u 2 , (1) a2
where p = jj is the cardinality of the alphabet . 1
3
0
2
Figure 1: The addresses of quadrants In image processing applications a four letter alphabet = f0; 1; 2; 3g will be used. Words of length k over will be interpreted as addresses of subsquares in the division of the unit square into 2k 2k subsquares as follows: Each letter refers to one quadrant as shown in Fig. 1. Word wa addresses the quadrant a of the subsquare addressed by w. A multiresolution function f : ?! [0; 1] de nes a sequence of gray-tone images with increasing resolutions: Its restriction to k de nes an image in resolution 2k 2k . The gray-tone intensity of a point in the subsquare addressed by w 2 k is f (w). The images at di erent resolutions are compatible if the multiresolution function is average preserving. In this case one can easily move from higher resolution to lower one by simply computing the averages of the intensities inside each subsquare. A multiresolution function f over = f0; 1; 2; 3g de nes an in nite resolution image ^f : [0; 1]2 ?! [0; 1] if the sequence fjk , k = 0; 1; 2; . . . of nite resolution images converges point-wise to f^. In a similar fashion, WFA over 2m letter alphabet are used to de ne functions [0; 1]m ! [0; 1] for all m 1. An m-state weighted nite automaton (WFA) A over alphabet is de ned by a row vector I A 2 IR1m (called the initial distribution), a column vector F A 2 IRm1 (the nal distribution), and weight matrices WaA 2 IRmm for all a 2 . The WFA A de nes a multiresolution function fA over by fA(a1a2 . . . ak ) = I A WaA1 WaA2 . . . WaAk F A: The WFA A is called average preserving if X A A (2) Wa F = p F A; a2
2
where p = jj is the cardinality of the alphabet . In other words, a WFA is average P preserving if its nal distribution is an eigenvector of a2 WaA corresponding to its eigenvalue jj. It is known (see [4]) that the multiresolution function computed by an average preserving WFA is average preserving, and that every average preserving multiresolution function computable by a WFA can be computed by an average preserving WFA. Analogously to WFA, an n-state weighted nite transducer (WFT) M from alphabet 1 into alphabet 2 is speci ed by 1. weight matrices Wa;b 2 IRnn for all a 2 1 [ f"g and b 2 2 [ f"g, 2. a row vector I 2 IR1n, called the initial distribution, and 3. a column vector F 2 IRn1 , called the nal distribution. The WFT M is called "-free if weight matrices W";" , Wa;" and W";b are zero matrices for all a 2 1 and b 2 2. The WFT M de nes function fM : 1 2 ?! IR, called weighted relation between 1 and 2, by fM (u; v) = I Wu;v F; for all u 2 1; v 2 2, where X (3) Wa1;b1 Wa2;b2 . . . Wak ;bk ; Wu;v = a1 . . . ak = u b1 . .. bk = v
if the sum converges. (If the sum does not converge, fM (u; v) remains unde ned.) In (3) the sum is taken over all decompositions of u an v into symbols ai 2 1 [ f"g and bi 2 2 [ f"g, respectively. In the special case of "-free transducers,
fM (a1a2 . . . ak ; b1b2 . . . bk ) = I Wa1;b1 Wa2;b2 . . . Wak;bk F; for a1a2 . . . ak 2 k1 ; b1b2 . . . bk 2 k2 , and fM (u; v) = 0; if juj 6= jvj. Let : 1 2 :?! IR be a weighted relation and f : 1 ?! IR a multiresolution function. The application of to f is the multiresolution function g = (f ) : 2 ?! IR over 2 de ned by X f (u)(u; v); for all v 2 2, g(v) = u21
provided the sum converges. The application M (f ) of WFT M to f is de ned as the application of the weighted relation fM to f , i.e. M (f ) = fM (f ). 3
If 1 = 2 = f0; 1; 2; 3g the weighted relation can be applied also on (integrable) in nite resolution images : [0; 1)2 ?! [0; 1]. Assume there exists an (unique) average preserving multiresolution function f such that f^ = . We de ne ( ) = d (f ), provided d (f ) exists and converges to an in nite resolution image (f ). The application of a WFT M to is de ned as the application of fM to . Lemma 1 WFT M is a linear operator IR1 ?! IR2 . In other words, M (r1f1 + r2f2) = r1M (f1) + r2M (f2); for all r1 ; r2 2 IR and f1; f2 : 1 ! IR. More generally, any weighted relation acts as a linear operator. 2 It follows naturally from Lemma 1 that weighted relations over four letter alphabet act as linear operators of in nite resolution images. In [2] the application of WFT M to WFA A was de ned. For simplicity we de ne it here only for "-free WFT. The application of an "-free WFT M to an m-state WFA A over alphabet 1 de ned by initial distribution I A, nal distribution F A and weight matrices WaA , a 2 1, is the mn-state WFA B = M (A) over alphabet 2 with initial distribution I B = I I A, nal distribution F B = F F A and weight matrices X WbB = Wa;b WaA; for all b 2 2. a21
Here, denotes the ordinary tensor product of matrices (called also Kronecker product or direct product), de ned as follows: Let T and Q be matrices of sizes s t and p q, respectively. Then their tensor product is the matrix 0 1 T 11Q T1tQ B .. ... C CA T Q=B @ . Ts1Q TstQ of size sp tq. Clearly, fB = M (fA ), i.e. the multiresolution function de ned by B is the same as the application of the WFT M to the multiresolution function computed by WFA A.
3 Average preserving WFT
Let the cardinalities of the alphabets be p = j1j and q = j2j. We call the "-free WFT average preserving if for all a 2 1 holds X (4) Wa;b F = pq F: b22 4
In other words, the nal distribution F is an eigenvector of matrices Pb22 Wa;b corresponding to eigenvalue qp , for all a 2 1. According to the next theorem the application of an average preserving WFT to an average preserving WFA is an average preserving WFA. Moreover, application to any average preserving multiresolution function is average preserving.
Theorem 1 Let M be an "-free WFT. M (A) is average preserving for every average
preserving WFA A if and only if M is average preserving. Moreover, if M is average preserving, M (f ) is average preserving for every average preserving multiresolution function f . Proof. Let p = j1 j and q = j2j be the cardinalities of the alphabets. (=)) : Let a 2 1 be arbitrary. Consider the average preserving one-state WFA A, with initial and nal distributions equal to 1, and with weight matrices WaA = p and WaA0 = 0 for a0 6= a. Then B = M (A) is the WFA with initial and nal distributions I and F of M , respectively, and with weight matrices WbB = p Wa;b for all b 2 2. The average preserveness condition (2) for WFA B is clearly equivalent to (4) for letter a. Since we assume the WFA B is average preserving regardless of the choice of a, (4) has to be satis ed for every a 2 1. In other words, WFT M is average preserving. ((=) : Assume M satis es (4) for every a 2 1. Let A be an average preserving WFA with initial and nal distributions I A and F A, and with weight matrices WaA for a 2 1. Let B = M (A) be the application of M to A. Then
X
b22
WbB F B =
Wa;b WaA F F A
X
a 2 1 b 2 2
0 X @X
1 = Wa;bF A WaA F A a21 b22 X X q A A q = F WaA F A F
W F = a p a21 a21 p = qp F pF A = qF B:
In other words, B = M (A) is average preserving. If M is average preserving and f : 1 ?! IR is an average preserving multiresolution function, then g = M (f ) satis es for every v 2 2 X X X g(vb) = f (ua) (IWu;v Wa;bF ) b22
b22 u 2 1 a 2 1
5
=
X u 2 1 a 2 1
0 1 X f (ua)IWu;v @ Wa;bF A b22
0 1 X @X = f (ua)A IWu;v qp F u21 a21 X f (u)IWu;v F = qg(v); = q u21
2
that is, g is average preserving.
The composition of weighted relations was de ned in [2]. Let us recall its de nition, as well as de nitions of some other operations. Let : (1 )? (2 )? ?! IR and : (1)? (2)? ?! IR be weighted relations. De ne (composition), + (sum), r (product with scalar r 2 IR), (concatenation) and + (catenation closure) as follows: X ( )(u; v) = (u; w)(w; v); w22
( + )(u; v) = (u; v) + (u; v); (r)(u; v) = r(X u; v); (u1; v1)(u2; v2); and ( )(u; v) = u = u1 u2 v = v1 v2
(+)(u; v) = (u; X v) + ( )(u; v) + ( )(u; v) + . . . = (u1; v1)(u2; v2) . . . (uk ; vk ): u = u1 . .. uk v = v1 .. . vk
In the case of composition it is assumed that 2 = 1, and in the cases of addition and concatenation that 1 = 1 and 2 = 2. The composition is a weighted relation between (1 ) and (2) while all others are between (1 ) and (2 ). In the de nition of the catenation closure the sum is taken over all decompositions of words u and v into subwords ui and vi including empty words. It is possible that the sum does not converge, in which case the catenation closure is not de ned. The sum in the de nition of composition does not always converge either. However, if the weighted relations are de ned by "-free WFT the composition is always de ned. The following formulas for the applications of the weighted relations to multiresolution function f : (1 )? ?! IR follow from the de nitions above: ( )(f ) = ((f )); 6
(a) Original 512 512, 8 bpp
(b) Regenerated by a WFA, 0.07346 bpp
Figure 2: Image Carol ( + )(f ) (r)(f ) (( )(f )) (w) + ( )(f ) (w)
= (f ) + (f ); = rX (f ); = ((f )) (w1) ((f )) (w2); and w=w1 w2
= ((f )) (w) + (( )(f )) (w) + (( )) (f )(w) + . . . X = ((f )) (w1) . . . ((f )) (wk ): w=w1 ...wk
Again, the catenation closure is de ned only if the sum converges (the sum is over all decompositions of w into subwords). Let us de ne next corresponding operations on "-free WFT. Let A (B ) be an nAstate (respectively nB -state) "-free WFT from alphabet A1 (B1 respectively) to A2 (B2 ) A with initial distribution I A (I B ), nal distribution F A (F B) and weight matrices Wa;b B for a 2 B and b 2 B , respectively). De ne new "-free for a 2 A1 and b 2 A2 (Wa;b 1 2 WFT A B , A + B , rA for r 2 IR, A B and A+ as follows: Composition : Assume A2 = B1 . The composition A B is the nA nB -state WFT initial distribution I A I B, nal distribution F A F B and weight from A1 to B2 with P matrices W = A W A W B for all a 2 A ; b 2 B . a;b
c22
a;c
c;b
1
2
Sum : Assume Ai = Bi for i = 1; 2. The sum A + B is the (nA + nB )-state WFT from 7
0,0:1 1,1:1
0.5,1 Y
2,0:1 3,1:1
2,2:1 3,3:1
j
0,2:1 1,3:1
0.5,1
(a) WFT Squeeze
(b) Image Squeeze(Carol)
Figure 3: Ane transformation Squeeze : (x; y) ! f( x2 ; y); ( x+1 2 ; y )g A1 to A2 with initial distribution I A+B , nal distribution F A+B and weight matrices A+B , a 2 A and b 2 A given by Wa;b 1 2 A ! F I A+B = I A I B ; F A+B = F B ; ! A W 0 A + B a;b Wa;b = 0 W B : a;b Multiplication by scalar r 2 IR : The WFT rA is as A except that the initial distribution vector I A is replaced by rI A. Concatenation : Assume Ai = Bi for i = 1; 2. The concatenation of A and B can be de ned as the (nA + nB )-state WFT with ("; ")-transitions given by ! A 0 I= I 0 ; F = FB ; ! A AI B ! W 0 0 F a;b : Wa;b = 0 W B ; W";" = 0 0 a;b This is equivalent to the "-free WFT A B with initial distribution I AB = I + IW";" = 8
0,0:1 2,2:1
1,0
Y
1,1:1 3,3:1 0,1:1 2,3:1
0,1:1 2,3:1
?
0,1
Y
0,1
6
1,0:1 3,2:1
1,0:1 3,2:1
j
0,0:1 2,2:1
(a) WFT
j
0,1
K
1,1:1 3,3:1
(b) Image (Carol)
Figure 4: Ane transformation : (x; y) ! (x; 23 y)
I A I AF AI B , nal distribution F AB = F and weight matrices A W A F AI B ! W A B a;b a;b Wa;b = Wa;b + Wa;bW";" = 0 ; B Wa;b
for all a 2 A1 and b 2 A2 . Catenation closure: The weighted relation fA+ is de ned only if jfA("; ")j = I AF A < 1. Assume this is the case, and denote t = 1=(1 ? I AF A). The catenation closure of the WFT A can be de ned as the nA -state WFT that di ers from A only in the weight matrix for ("; ")-transitions: W";" = F AI A. An equivalent "+ + A free WFT A is speci ed+by initial distribution I = tI A, nal distribution F A+ = F A A = W A + tW A F A I A. and weight matrices Wa;b a;b a;b According to the following theorem the operations de ned on WFT and weighted relations are compatible. Its proof is straightforward. Theorem 2 For "-free WFT A and B holds fAB = fA fB , fA+B = fA + fB , frA = rfA 9
(a) ( + )(Carol)
(b) ( + )3(Carol)
Figure 5: Application of ( + ) to image Carol for all r 2 IR, fAB = fA fB and fA+ = fA+ . In the case of catenation closure we naturally assume that fA+ is de ned. 2
Our next theorem states that the class of average preserving WFT is closed under the operations of composition, addition and multiplication by a scalar.
Theorem 3 Let A and B be average preserving "-free WFT. Then A B , A + B and
rA are average preserving as well.
Proof. Straightforward computations verify (4) in all cases.
2
4 Examples Now, we will demonstrate that WFT can implement almost every useful linear operator. It has been shown in [2] that every ane transformation on IR2 is realized by a WFT. We give few examples of (piecewise) ane transformations restricted to [0; 1]2. We will illustrate most our WFT by mapping image Carol shown in Fig. 2. The original, resolution 512 512 8 bits per pixel, is in Fig. 2(a). The image in Fig. 2(b) is regenerated by WFA stored in 2406 bytes (109 compression 0.07346 bpp). 10
0,0:1 3,3:1
*
0,1
1,1:1
0,0:1
1,1:1 2,2:1
",0:1 ",3:1
*
*
0,1
0,1
0,1
0,1
id
3,3:1
1,0
1,1:1 2,2:1
id:
2,2:1
0,1
id
(a) WFT
0,0:1
-
,3:1
1,0
0,0:1 3,3:1
,1:1 ,2:1
,2:1
0,1
id
(b) WFT
,3:1
id:
,0:1
0,0:1 1,1:1 2,2:1 3,3:1
0,1
id
0,1
1,1:1
" " I @ ? @@ @ @@" ?"? " @ @@ ?? @R ? @@ ? ? ? " " " ? @ ? @@ ?? " ? R ? " " ",1:1
,0:1
*
-
3,3:1
,1:1
",1:1 ",2:1
I @ ? @@ @@ ? @@ @ ?? @R @ ? ? @ ? ? @ ? @@ ? ?? ? @R ? ? 2,2:1
0,1
0,0:1 1,1:1 2,2:1 3,3:1
,2:1
,0:1 ,3:1
Figure 6: WFT for \restricted identity" and for \fractal copies" 11
shift right: 0,0:1 1,1:1
shift left: 0,0:1 1,1:1
* 1,0
- 1,1
1,0:1
* 1,0
1,0:1
- 1,1
0,1:1
- 1,1
1,0:1 3,2:1
shift up in two dimensions: 0,0:1 1,1:1 2,2:1 3,3:1
0,1:1
* 1,0
0,1:1 2,3:1
Figure 7: WFT computing circular shifts by one pixel We display WFT using similar diagrams as are used for nite automata. States are represented by circles, the initial and nal distribution is shown inside the circles. If (Wa;b)i;j 6= 0, then there is an edge from state i to state j labeled by a; b :(Wa;b)i;j . WFA Squeeze shown in Fig. 3(a) implements the sum of two ane transformations x1 = x2 ; y1 = y and x2 = x +2 1 ; y2 = y: The image Squeeze(Carol) is shown in Fig. 3(b). The WFT ' that implements the ane transformation ' : (x; y) ! (x; 23 y) is shown in Fig. 4(a), and the image ' (Carol) in the Fig. 4(b). Next we will do complicated cutting and pasting. WFT shown in Fig. 6(a) copies the portion of the image in the triangular half of each quadrant closer to the center and leaves zero in the rest. WFT shown in Fig. 6(b) makes \diminishing copies" of the input image in the other halfs of the quadrants. The application of the sum + to image Carol is shown in Fig. 5. The technique used in the examples allows to suggest the following de nition and theorem. 12
0,0:1 1,1:1
?? ??1,0:1 ? 1 * 1, 2 @@ @0,1:1 @ @R
1, 14
1, 14
0,1:1
1,0:1
Figure 8: A WFT computing the lter F (x) = 41 f (x ? h) + 21 f (x) + 14 f (x + h)
De nition 1 A transformation composed from ane transformations by cutting (along \rational" lines), pasting and addition is called piecewise ane transformation.
Theorem 4 Every piecewise ane transformation can be implemented by a WFT. Proof outline. It has been shown in [2] that every ane transformation (restricted to [0; 1]2) can be implemented by a WFA. A practice of \rational" cutting, i.e. the restriction to a regular set can, clearly, be implemented by a WFT. Pasting is a special case of addition of WFT. Hence by the closure of WFT under composition and addition we can implement every piecewise ane transformation. Actually, using the operation of concatenation we can paste in nite number of copies as long as the in nite copying can be expressed by a regular set (WFA if the grayness is not uniform). 2 Note that every ane transformation is implemented by an average preserving WFT. This is not necessary case for piecewise ane transformation, however, every piecewise ane transformation can be implemented by a WFT in which all states visited more than uniformly bounded number of times satis es the average preserving condition (4). Many interesting WFT can be designed using the technique of value shifting. In one dimension, the shift by one pixel (one step in a function table) for resolution 2n ; n 1 requires to move the value at address w01r to the address w10r for 0 r n ? 1 and w 2 n?r?1 . Therefore, somewhat surprisingly, we easily design a WFT with only 2 states which performs this shift. By appropriately choosing the initial distribution we can make the shift circular. WFT computing various shifts are shown in Fig. 7. Since WFT are closed under composition and addition we, clearly, can design a WFT that computes any linear combination of the values of each cell and any xed nite set of 13
d f (x) : dx
0,0:2 1,1:2
@f (x;y) : @x
0,0:2 1,1:2 2,2:2 3,3:2
* 1,-1 * 1,-1
1,0:2 2,0:2 3,1:2
- 1,1
0,1:2
- 1,1
0,2:2 1,3:2
1,0:2 0,0:2 3,2:2 1,1:2 0,1:2 1,-1 1,1 * 2,2:2 2,3:2 3,3:2 Figure 9: WFT computing derivative and partial derivatives
@f (x;y) : @y
its neighbors. Examples of such WFT are a WFT that simulates the moves of a knight on a 2n 2n chess-board, WFT that implements any type of low or high pass lter (one such lter is shown in Fig. 8) or WFT Di which for any nite resolution computes f (x+h)?f (x) for table step h. Thus WFT Di computes the derivative df (x) in the limit h dx (in nite resolution). WFT Di and similar WFT that compute the partial derivates for functions of two variables are shown in Fig. 9. In Fig. 10 we show the sign of the partial derivatives of Carol. Any positive value is represented by white color, negative by black. Computing of integrals requires to compute linear combinations of values from an unbounded number of neighbors but that can be done, too. WFT shown at the top of Fig. 11, for each resolution, computes h(f (0) + f (h) + f (2h) + . . .+ f (x)) where h is the table step (distance of two neighboring pixels). Thus for the in nite resolution WFT int computes Zx f (x)dx = hlim h(f (0) + f (h) + f (2h) + . . . + f (x)) : !0 0 R R WFT computing (in the limit) 0y 0x f (x; y)dxdy is shown at the bottom of Fig. 11. In all the examples above the same small WFT computes the desired transformation for every resolution and for the in nite resolution as well. For the shifts the limiting case is the identity transformation. However, it is easy to give an average preserving WFT which does not converge to any mapping on in nite resolution. Consider one state 14
@ Carol(x;y) @x
@ Carol(x;y) @ Carol(x;y) @y @x@y Figure 10: The partial derivatives of Carol
WFT with I=F=1, and transitions (0,0:1),(1,0:1), (2,2:1) and (3,2:1).
Theorem 5 Let = f0; 1g and A be a WFA over , Then we can construct WFA B
such that (i) f^B (x) = df^dxA(x) for all x 2 [0; 1] for which df^dxA(x) exists. (ii) f^B (x) = R0x fA (t) dt if the Rieman integral exists.
Proof. Follows from the Examples above and the closure of WFA under WFT [2].
2
Note that (ii) is an extension of a result in [3] where this result was shown for the restricted case of the average preserving WFA. However, if WFA A has n states the construction in [3] yields B with n + 1 states while the application of our two state integrating WFA yields WFA B with 2n states. Theorem 6 Let = f0; 1; 2; 3g and A be a WFA over with f^A : [0; 1]2 ! IR. Then we can construct WFA B such that (i) f^B (x; y) = @@f^iAx@(x;yj y ) for all (x; y) 2 [0; 1]2 for which the derivative exists and all i; j 1. (ii) f^B (x; y) = R0x f^A(t; y) dt for all x 2 [0; 1] for which the Rieman integral exists. (iii) f^B (x; y) = R0y f^A(x; t) dt for all y 2 [0; 1] for which the Rieman integral exists. 15
(iv) f^B (x; y) = R0xR0y f^A (t; s) ds dt for all (x; y) 2 [0; 1]2 for which the Rieman integral exists.
Proof. Follows from the Examples above, the closure of WFA under composition and the closure of WFA under WFT [2].
2
WFT Mallat shown in Fig. 12 computes the coecients of the discrete Haar wavelet transform for any nite resolution and presents them in the Mallat form [8]. It computes the continuous Haar wavelet transform for the in nite resolution. The image Mallat(Carol) is shown in Fig. 13 with the contrast increased. In order to display better all the coecients we compute them unscaled that is use wavelets which are not orthonormal. To get the orthonormal case it is sucient just to adjust the weights of the " transitions of our WFT. All the examples of image transformations here were produced using a menu-driven X-windows based system wftx implemented by P.Rajcani. In wftx images are represented either in pixel form or by WFA. The conversions between these representations are implemented using the WFA inference and the WFA decoding algorithms from [5, 6]. All the operations on images, WFA and WFT, except for the concatenation closure are implemented in wftx. In particular, a WFT can be applied to an image in pixel form (resolution 2n 2n ; n 1) and produce again an image in possibly di erent resolution, or a WFT can be applied to a WFA and produce a WFA. Any of the WFA computing integrals is a typical example of a WFA which has a small number of states but it is highly nondeterministic. For such a WFT, it is much more ecient to apply it to an image in the WFA representation even if it requires converting the image to the WFA representation and then the WFA decoding.
5 Conclusions In the previous section we have shown that many relatively complicated image transformations can be expressed by simple WFT. Since there exists an ecient implementation of WFT, we can use WFT as a powerful design tool for image transformation. In addition, using the wftx system, any WFT can be applied to an image either in the pixel form or in the WFA-compressed form. Due to this property, we can process compressed images without decoding them. It should also be noted that WFT provide a rigorous mathematical description of image transformations. WFA and WFT can also be used with alphabets of di erent size than 22 = 4. Using an alphabet with 32 = 9 letters enables us to manipulate images of size 3n 3n , using 3 2 = 6 letters images of size 3n 2n, e.t.c. 16
If image approximations are sucient, we can process images of arbitrary rectangular size by squeezing/stretching them rst to the closest size 2n 2n and after manipulation unsqueezing/unstretching them back to the original size. This method has been implemented in the wftx system.
17
Z x 0
f (t)dt
:
*
0,0:0:5 1,1:0:5
Z yZ x 0
0
-
1,1
1 4 1 0,1: 4 1 1,0: 4 1 1,1: 4 0,0:
f (x; y )dx dy
0,1:0:5
:
1 4 1 1,1: 4 1 2,2: 4 1 3,3: 4
*
1,1
1 4 1 0,3: 4 1 1,2: 4 1 1,3: 4
1,3:
0,1
1 4 1 0,2: 4 1 2,0: 4 1 1,1: 4 0,0:
K
0,1:
0,3: 2,1: 2,3:
4
1 4 1 1,3: 4 1 3,1: 4 1 3,3: 4
2,2:
Figure 11: WFT computing integrals 18
1 4 1 0,1: 4 1 0,2: 4 1 0,3: 4 1 1,0: 4 1 1,1: 4 1 1,2: 4 1 1,3: 4 1 2,0: 4 1 2,1: 4 1 2,2: 4 1 2,3: 4 1 3,0: 4 1 3,1: 4 1 3,2: 4 1 3,3: 4
0,0:
0,2:
0,3:
0,2:
1,0:0:5 1,1:0:5
1 4 1 2,3: 4 1 3,2: 4 1 3,3: 4
?? @@ @@ ?? @@ ?? @@ ?? ? 1 4 -R@ @@ ?? @@ ?? ? 1 1@ ? 4 @ 4 1 1 ? @@ 4 4 ? 1 @R ? 4 1 1 4 1 2,3: 4 0,1:
0,0:0:5 0,1:0:5
2,2:
0,1
0,0:
0,1
0,1
",0:1
?
* 1,0 ?? @@
@@ ",2:1 @ ",3:1 ? @@ ? ? @ ? @@R 0,0:1 0,0:1 ? 1,1:1 ? 1,1:1 * 0,0 2,2:1 * 0,0 2,2:1 * 0,0 @@ 3,3:1 ? 3,3:1 ? 0,":1 @@ ? 1,":-1 ? @ ? 2,":1 ? 0,":-1 @ 3,":-1 ? 0,":-1 1,":-1 @@ 1,":1 ? 2,":1 2,":1 @@R ? ? ? 3,":1 3,":-1 ",1:1??
0,0:1 1,1:1 2,2:1 3,3:1
0,1
Figure 12: WFT computing Haar wavelet coecients in Mallat form
Figure 13: The image Mallat(Carol) 19
References [1] K. Culik II and S. Dube, Balancing Order and Chaos in Image Generation, Computers and Graphics 17, 4, 465-486 (1993). [2] K. Culik II and I. Fris, Weighted Finite Transducers in Image Processing. Discrete Applied Mathematics 58, 223-237 (1995). [3] K. Culik II and J. Karhumaki, Automata Computing Real Functions, SIAM J. on Computing 23, 4, 789-814 (1994). [4] K. Culik II and J. Kari, Image Compression Using Weighted Finite Automata, Computers and Graphics 17, 3, 305-313 (1993). [5] K. Culik II and J. Kari, Image-Data Compression Using Edge-Optimizing Algorithm for WFA Inference, Journal of Information Processing and Management 30, 6, 829838 (1994). [6] K. Culik II and J. Kari, Ecient Inference Algorithm for Weighted Finite Automata, in Fractal Image Encoding and Compression, ed. Y.Fisher, pp. 243-258, SpringerVerlag (1994). [7] K. Culik II and P. Rajcani, Iterative Weighted Finite Transductions, Acta Informatica, to appear. [8] R. A. DeVore, B. Jawerth and B. J. Lucier, Image Compression through Wavelet Transform Coding, IEEE Transactions of Information Theory 38, 719-746 (1992). [9] S. Eilenberg, Automata, Languages and Machines, Vol. A, Academic Press, New York (1974).
20
Department of Computer Science University of South Carolina Columbia, S.C. 29208
Jarkko Kari y
Academy of Finland
Abstract
Weighted nite automata (WFA) have been introduced as devices for computing real functions on [0; 1]n. The main motivation has been to generate functions on [0; 1] [0; 1] interpreted as gray-tone images. Weighted nite transducers (WFT) are nite state devices that serve as a powerful tool for describing and implementing a large variety of image transformations and more generally linear operators on real functions. Here we show new results on WFT and demonstrate that WFT are indeed an excellent tool for image manipulation and more generally for function transformation. We note that every WFA transformation is a linear operator and show that most of the interesting linear operators on real functions (on [0; 1]2) can be easily implemented by WFT. We give a number of examples that include ane transformations, a low-pass lter, wavelet transform, (partial) derivatives, simple and multiple integrals. Since the family of WFA-functions is constructively closed under WFT, each of our examples is actually a proof of a theorem stating that for each WFA A there e ectively exists another WFA B that computes the integral (or other transformations) of the function de ned by A.
Research was supported by the National Science Foundation under Grant No. CCR-9202396. Preliminary version of this paper has been presented at ICALP 95. yThe address of the second author is: Mathematics Department, University of Turku, 20500 Turku, Finland
0
1 Introduction Weighted nite automata (WFA) have been introduced in [3, 4] as devices for computing real functions on [0; 1]n . The main motivation has been to generate functions on [0; 1] [0; 1] interpreted as gray-tone images. In [4, 5] (see also [6]) we developed inference algorithms for WFA. Using the algorithm from [5] we can eciently encode any image (digitalized photograph) by a WFA. One of the best image compression method has been developed on this basis. In [2] the generalized k-tape WFA have been introduced and in particular, the 2tape WFA called weighted nite transducers (WFT) have been studied and shown to perform a number of simple operations on images. When considered as only de ning mappings on nite words WFT are a special case of rational transducers [9]. In [7] the iterative WFT have been studied and shown to be strictly more powerful image generators than the mutual recursive function systems (MRFS), see e.g. [1]. Here we show new results on WFT and demonstrate that WFT are indeed an excellent tool for image manipulation and more generally for function transformation. We note that every WFA transformation is a linear operator and show that most of the interesting linear operators on real functions (on [0; 1]2) can be easily implemented by the WFT. We give a number of examples that include ane transformations, a low-pass lter, wavelet transform, (partial) derivatives, simple and multiple integrals. Since the family of WFA-functions is constructively closed under WFT [2], each of our examples is actually a proof of a theorem stating that for each WFA A there e ectively exists another WFA B that computes the integral (or other transformations) of the function de ned by A. In the case of integrals this has been shown in [3] for average preserving WFA. We extend this result to arbitrary WFA and to multiple integrals. In Section 2 we introduce an automata-theoretic notation for multiresolution images and weighted nite automata and transducers. In Section 3 we introduce average preserving WFT and show that average preserving WFA are closed under average preserving WFT. This is important since an average preserving WFA generates an average preserving function that is a well de ned multiresolution image. We show that average preserving WFT are closed under the operations of composition, addition, and multiplication by a constant. In Section 4 we demonstrate the surprising ability and exibility of WFT to implement image transformations and linear function operators. We show that every piecewise ane transformation can be implemented by WFT. The crucial, somewhat tricky example, of a WFT is the one that shifts an image by one pixel for any nite resolution. It leads us to implementation of lters, (partial) derivatives, and other linear operators.
1
2 Multiresolution functions, WFA and WFT
A multiresolution function f over alphabet is a function ?! IR. It is average preserving if X f (ua) = p f (u); for all u 2 , (1) a2
where p = jj is the cardinality of the alphabet . 1
3
0
2
Figure 1: The addresses of quadrants In image processing applications a four letter alphabet = f0; 1; 2; 3g will be used. Words of length k over will be interpreted as addresses of subsquares in the division of the unit square into 2k 2k subsquares as follows: Each letter refers to one quadrant as shown in Fig. 1. Word wa addresses the quadrant a of the subsquare addressed by w. A multiresolution function f : ?! [0; 1] de nes a sequence of gray-tone images with increasing resolutions: Its restriction to k de nes an image in resolution 2k 2k . The gray-tone intensity of a point in the subsquare addressed by w 2 k is f (w). The images at di erent resolutions are compatible if the multiresolution function is average preserving. In this case one can easily move from higher resolution to lower one by simply computing the averages of the intensities inside each subsquare. A multiresolution function f over = f0; 1; 2; 3g de nes an in nite resolution image ^f : [0; 1]2 ?! [0; 1] if the sequence fjk , k = 0; 1; 2; . . . of nite resolution images converges point-wise to f^. In a similar fashion, WFA over 2m letter alphabet are used to de ne functions [0; 1]m ! [0; 1] for all m 1. An m-state weighted nite automaton (WFA) A over alphabet is de ned by a row vector I A 2 IR1m (called the initial distribution), a column vector F A 2 IRm1 (the nal distribution), and weight matrices WaA 2 IRmm for all a 2 . The WFA A de nes a multiresolution function fA over by fA(a1a2 . . . ak ) = I A WaA1 WaA2 . . . WaAk F A: The WFA A is called average preserving if X A A (2) Wa F = p F A; a2
2
where p = jj is the cardinality of the alphabet . In other words, a WFA is average P preserving if its nal distribution is an eigenvector of a2 WaA corresponding to its eigenvalue jj. It is known (see [4]) that the multiresolution function computed by an average preserving WFA is average preserving, and that every average preserving multiresolution function computable by a WFA can be computed by an average preserving WFA. Analogously to WFA, an n-state weighted nite transducer (WFT) M from alphabet 1 into alphabet 2 is speci ed by 1. weight matrices Wa;b 2 IRnn for all a 2 1 [ f"g and b 2 2 [ f"g, 2. a row vector I 2 IR1n, called the initial distribution, and 3. a column vector F 2 IRn1 , called the nal distribution. The WFT M is called "-free if weight matrices W";" , Wa;" and W";b are zero matrices for all a 2 1 and b 2 2. The WFT M de nes function fM : 1 2 ?! IR, called weighted relation between 1 and 2, by fM (u; v) = I Wu;v F; for all u 2 1; v 2 2, where X (3) Wa1;b1 Wa2;b2 . . . Wak ;bk ; Wu;v = a1 . . . ak = u b1 . .. bk = v
if the sum converges. (If the sum does not converge, fM (u; v) remains unde ned.) In (3) the sum is taken over all decompositions of u an v into symbols ai 2 1 [ f"g and bi 2 2 [ f"g, respectively. In the special case of "-free transducers,
fM (a1a2 . . . ak ; b1b2 . . . bk ) = I Wa1;b1 Wa2;b2 . . . Wak;bk F; for a1a2 . . . ak 2 k1 ; b1b2 . . . bk 2 k2 , and fM (u; v) = 0; if juj 6= jvj. Let : 1 2 :?! IR be a weighted relation and f : 1 ?! IR a multiresolution function. The application of to f is the multiresolution function g = (f ) : 2 ?! IR over 2 de ned by X f (u)(u; v); for all v 2 2, g(v) = u21
provided the sum converges. The application M (f ) of WFT M to f is de ned as the application of the weighted relation fM to f , i.e. M (f ) = fM (f ). 3
If 1 = 2 = f0; 1; 2; 3g the weighted relation can be applied also on (integrable) in nite resolution images : [0; 1)2 ?! [0; 1]. Assume there exists an (unique) average preserving multiresolution function f such that f^ = . We de ne ( ) = d (f ), provided d (f ) exists and converges to an in nite resolution image (f ). The application of a WFT M to is de ned as the application of fM to . Lemma 1 WFT M is a linear operator IR1 ?! IR2 . In other words, M (r1f1 + r2f2) = r1M (f1) + r2M (f2); for all r1 ; r2 2 IR and f1; f2 : 1 ! IR. More generally, any weighted relation acts as a linear operator. 2 It follows naturally from Lemma 1 that weighted relations over four letter alphabet act as linear operators of in nite resolution images. In [2] the application of WFT M to WFA A was de ned. For simplicity we de ne it here only for "-free WFT. The application of an "-free WFT M to an m-state WFA A over alphabet 1 de ned by initial distribution I A, nal distribution F A and weight matrices WaA , a 2 1, is the mn-state WFA B = M (A) over alphabet 2 with initial distribution I B = I I A, nal distribution F B = F F A and weight matrices X WbB = Wa;b WaA; for all b 2 2. a21
Here, denotes the ordinary tensor product of matrices (called also Kronecker product or direct product), de ned as follows: Let T and Q be matrices of sizes s t and p q, respectively. Then their tensor product is the matrix 0 1 T 11Q T1tQ B .. ... C CA T Q=B @ . Ts1Q TstQ of size sp tq. Clearly, fB = M (fA ), i.e. the multiresolution function de ned by B is the same as the application of the WFT M to the multiresolution function computed by WFA A.
3 Average preserving WFT
Let the cardinalities of the alphabets be p = j1j and q = j2j. We call the "-free WFT average preserving if for all a 2 1 holds X (4) Wa;b F = pq F: b22 4
In other words, the nal distribution F is an eigenvector of matrices Pb22 Wa;b corresponding to eigenvalue qp , for all a 2 1. According to the next theorem the application of an average preserving WFT to an average preserving WFA is an average preserving WFA. Moreover, application to any average preserving multiresolution function is average preserving.
Theorem 1 Let M be an "-free WFT. M (A) is average preserving for every average
preserving WFA A if and only if M is average preserving. Moreover, if M is average preserving, M (f ) is average preserving for every average preserving multiresolution function f . Proof. Let p = j1 j and q = j2j be the cardinalities of the alphabets. (=)) : Let a 2 1 be arbitrary. Consider the average preserving one-state WFA A, with initial and nal distributions equal to 1, and with weight matrices WaA = p and WaA0 = 0 for a0 6= a. Then B = M (A) is the WFA with initial and nal distributions I and F of M , respectively, and with weight matrices WbB = p Wa;b for all b 2 2. The average preserveness condition (2) for WFA B is clearly equivalent to (4) for letter a. Since we assume the WFA B is average preserving regardless of the choice of a, (4) has to be satis ed for every a 2 1. In other words, WFT M is average preserving. ((=) : Assume M satis es (4) for every a 2 1. Let A be an average preserving WFA with initial and nal distributions I A and F A, and with weight matrices WaA for a 2 1. Let B = M (A) be the application of M to A. Then
X
b22
WbB F B =
Wa;b WaA F F A
X
a 2 1 b 2 2
0 X @X
1 = Wa;bF A WaA F A a21 b22 X X q A A q = F WaA F A F
W F = a p a21 a21 p = qp F pF A = qF B:
In other words, B = M (A) is average preserving. If M is average preserving and f : 1 ?! IR is an average preserving multiresolution function, then g = M (f ) satis es for every v 2 2 X X X g(vb) = f (ua) (IWu;v Wa;bF ) b22
b22 u 2 1 a 2 1
5
=
X u 2 1 a 2 1
0 1 X f (ua)IWu;v @ Wa;bF A b22
0 1 X @X = f (ua)A IWu;v qp F u21 a21 X f (u)IWu;v F = qg(v); = q u21
2
that is, g is average preserving.
The composition of weighted relations was de ned in [2]. Let us recall its de nition, as well as de nitions of some other operations. Let : (1 )? (2 )? ?! IR and : (1)? (2)? ?! IR be weighted relations. De ne (composition), + (sum), r (product with scalar r 2 IR), (concatenation) and + (catenation closure) as follows: X ( )(u; v) = (u; w)(w; v); w22
( + )(u; v) = (u; v) + (u; v); (r)(u; v) = r(X u; v); (u1; v1)(u2; v2); and ( )(u; v) = u = u1 u2 v = v1 v2
(+)(u; v) = (u; X v) + ( )(u; v) + ( )(u; v) + . . . = (u1; v1)(u2; v2) . . . (uk ; vk ): u = u1 . .. uk v = v1 .. . vk
In the case of composition it is assumed that 2 = 1, and in the cases of addition and concatenation that 1 = 1 and 2 = 2. The composition is a weighted relation between (1 ) and (2) while all others are between (1 ) and (2 ). In the de nition of the catenation closure the sum is taken over all decompositions of words u and v into subwords ui and vi including empty words. It is possible that the sum does not converge, in which case the catenation closure is not de ned. The sum in the de nition of composition does not always converge either. However, if the weighted relations are de ned by "-free WFT the composition is always de ned. The following formulas for the applications of the weighted relations to multiresolution function f : (1 )? ?! IR follow from the de nitions above: ( )(f ) = ((f )); 6
(a) Original 512 512, 8 bpp
(b) Regenerated by a WFA, 0.07346 bpp
Figure 2: Image Carol ( + )(f ) (r)(f ) (( )(f )) (w) + ( )(f ) (w)
= (f ) + (f ); = rX (f ); = ((f )) (w1) ((f )) (w2); and w=w1 w2
= ((f )) (w) + (( )(f )) (w) + (( )) (f )(w) + . . . X = ((f )) (w1) . . . ((f )) (wk ): w=w1 ...wk
Again, the catenation closure is de ned only if the sum converges (the sum is over all decompositions of w into subwords). Let us de ne next corresponding operations on "-free WFT. Let A (B ) be an nAstate (respectively nB -state) "-free WFT from alphabet A1 (B1 respectively) to A2 (B2 ) A with initial distribution I A (I B ), nal distribution F A (F B) and weight matrices Wa;b B for a 2 B and b 2 B , respectively). De ne new "-free for a 2 A1 and b 2 A2 (Wa;b 1 2 WFT A B , A + B , rA for r 2 IR, A B and A+ as follows: Composition : Assume A2 = B1 . The composition A B is the nA nB -state WFT initial distribution I A I B, nal distribution F A F B and weight from A1 to B2 with P matrices W = A W A W B for all a 2 A ; b 2 B . a;b
c22
a;c
c;b
1
2
Sum : Assume Ai = Bi for i = 1; 2. The sum A + B is the (nA + nB )-state WFT from 7
0,0:1 1,1:1
0.5,1 Y
2,0:1 3,1:1
2,2:1 3,3:1
j
0,2:1 1,3:1
0.5,1
(a) WFT Squeeze
(b) Image Squeeze(Carol)
Figure 3: Ane transformation Squeeze : (x; y) ! f( x2 ; y); ( x+1 2 ; y )g A1 to A2 with initial distribution I A+B , nal distribution F A+B and weight matrices A+B , a 2 A and b 2 A given by Wa;b 1 2 A ! F I A+B = I A I B ; F A+B = F B ; ! A W 0 A + B a;b Wa;b = 0 W B : a;b Multiplication by scalar r 2 IR : The WFT rA is as A except that the initial distribution vector I A is replaced by rI A. Concatenation : Assume Ai = Bi for i = 1; 2. The concatenation of A and B can be de ned as the (nA + nB )-state WFT with ("; ")-transitions given by ! A 0 I= I 0 ; F = FB ; ! A AI B ! W 0 0 F a;b : Wa;b = 0 W B ; W";" = 0 0 a;b This is equivalent to the "-free WFT A B with initial distribution I AB = I + IW";" = 8
0,0:1 2,2:1
1,0
Y
1,1:1 3,3:1 0,1:1 2,3:1
0,1:1 2,3:1
?
0,1
Y
0,1
6
1,0:1 3,2:1
1,0:1 3,2:1
j
0,0:1 2,2:1
(a) WFT
j
0,1
K
1,1:1 3,3:1
(b) Image (Carol)
Figure 4: Ane transformation : (x; y) ! (x; 23 y)
I A I AF AI B , nal distribution F AB = F and weight matrices A W A F AI B ! W A B a;b a;b Wa;b = Wa;b + Wa;bW";" = 0 ; B Wa;b
for all a 2 A1 and b 2 A2 . Catenation closure: The weighted relation fA+ is de ned only if jfA("; ")j = I AF A < 1. Assume this is the case, and denote t = 1=(1 ? I AF A). The catenation closure of the WFT A can be de ned as the nA -state WFT that di ers from A only in the weight matrix for ("; ")-transitions: W";" = F AI A. An equivalent "+ + A free WFT A is speci ed+by initial distribution I = tI A, nal distribution F A+ = F A A = W A + tW A F A I A. and weight matrices Wa;b a;b a;b According to the following theorem the operations de ned on WFT and weighted relations are compatible. Its proof is straightforward. Theorem 2 For "-free WFT A and B holds fAB = fA fB , fA+B = fA + fB , frA = rfA 9
(a) ( + )(Carol)
(b) ( + )3(Carol)
Figure 5: Application of ( + ) to image Carol for all r 2 IR, fAB = fA fB and fA+ = fA+ . In the case of catenation closure we naturally assume that fA+ is de ned. 2
Our next theorem states that the class of average preserving WFT is closed under the operations of composition, addition and multiplication by a scalar.
Theorem 3 Let A and B be average preserving "-free WFT. Then A B , A + B and
rA are average preserving as well.
Proof. Straightforward computations verify (4) in all cases.
2
4 Examples Now, we will demonstrate that WFT can implement almost every useful linear operator. It has been shown in [2] that every ane transformation on IR2 is realized by a WFT. We give few examples of (piecewise) ane transformations restricted to [0; 1]2. We will illustrate most our WFT by mapping image Carol shown in Fig. 2. The original, resolution 512 512 8 bits per pixel, is in Fig. 2(a). The image in Fig. 2(b) is regenerated by WFA stored in 2406 bytes (109 compression 0.07346 bpp). 10
0,0:1 3,3:1
*
0,1
1,1:1
0,0:1
1,1:1 2,2:1
",0:1 ",3:1
*
*
0,1
0,1
0,1
0,1
id
3,3:1
1,0
1,1:1 2,2:1
id:
2,2:1
0,1
id
(a) WFT
0,0:1
-
,3:1
1,0
0,0:1 3,3:1
,1:1 ,2:1
,2:1
0,1
id
(b) WFT
,3:1
id:
,0:1
0,0:1 1,1:1 2,2:1 3,3:1
0,1
id
0,1
1,1:1
" " I @ ? @@ @ @@" ?"? " @ @@ ?? @R ? @@ ? ? ? " " " ? @ ? @@ ?? " ? R ? " " ",1:1
,0:1
*
-
3,3:1
,1:1
",1:1 ",2:1
I @ ? @@ @@ ? @@ @ ?? @R @ ? ? @ ? ? @ ? @@ ? ?? ? @R ? ? 2,2:1
0,1
0,0:1 1,1:1 2,2:1 3,3:1
,2:1
,0:1 ,3:1
Figure 6: WFT for \restricted identity" and for \fractal copies" 11
shift right: 0,0:1 1,1:1
shift left: 0,0:1 1,1:1
* 1,0
- 1,1
1,0:1
* 1,0
1,0:1
- 1,1
0,1:1
- 1,1
1,0:1 3,2:1
shift up in two dimensions: 0,0:1 1,1:1 2,2:1 3,3:1
0,1:1
* 1,0
0,1:1 2,3:1
Figure 7: WFT computing circular shifts by one pixel We display WFT using similar diagrams as are used for nite automata. States are represented by circles, the initial and nal distribution is shown inside the circles. If (Wa;b)i;j 6= 0, then there is an edge from state i to state j labeled by a; b :(Wa;b)i;j . WFA Squeeze shown in Fig. 3(a) implements the sum of two ane transformations x1 = x2 ; y1 = y and x2 = x +2 1 ; y2 = y: The image Squeeze(Carol) is shown in Fig. 3(b). The WFT ' that implements the ane transformation ' : (x; y) ! (x; 23 y) is shown in Fig. 4(a), and the image ' (Carol) in the Fig. 4(b). Next we will do complicated cutting and pasting. WFT shown in Fig. 6(a) copies the portion of the image in the triangular half of each quadrant closer to the center and leaves zero in the rest. WFT shown in Fig. 6(b) makes \diminishing copies" of the input image in the other halfs of the quadrants. The application of the sum + to image Carol is shown in Fig. 5. The technique used in the examples allows to suggest the following de nition and theorem. 12
0,0:1 1,1:1
?? ??1,0:1 ? 1 * 1, 2 @@ @0,1:1 @ @R
1, 14
1, 14
0,1:1
1,0:1
Figure 8: A WFT computing the lter F (x) = 41 f (x ? h) + 21 f (x) + 14 f (x + h)
De nition 1 A transformation composed from ane transformations by cutting (along \rational" lines), pasting and addition is called piecewise ane transformation.
Theorem 4 Every piecewise ane transformation can be implemented by a WFT. Proof outline. It has been shown in [2] that every ane transformation (restricted to [0; 1]2) can be implemented by a WFA. A practice of \rational" cutting, i.e. the restriction to a regular set can, clearly, be implemented by a WFT. Pasting is a special case of addition of WFT. Hence by the closure of WFT under composition and addition we can implement every piecewise ane transformation. Actually, using the operation of concatenation we can paste in nite number of copies as long as the in nite copying can be expressed by a regular set (WFA if the grayness is not uniform). 2 Note that every ane transformation is implemented by an average preserving WFT. This is not necessary case for piecewise ane transformation, however, every piecewise ane transformation can be implemented by a WFT in which all states visited more than uniformly bounded number of times satis es the average preserving condition (4). Many interesting WFT can be designed using the technique of value shifting. In one dimension, the shift by one pixel (one step in a function table) for resolution 2n ; n 1 requires to move the value at address w01r to the address w10r for 0 r n ? 1 and w 2 n?r?1 . Therefore, somewhat surprisingly, we easily design a WFT with only 2 states which performs this shift. By appropriately choosing the initial distribution we can make the shift circular. WFT computing various shifts are shown in Fig. 7. Since WFT are closed under composition and addition we, clearly, can design a WFT that computes any linear combination of the values of each cell and any xed nite set of 13
d f (x) : dx
0,0:2 1,1:2
@f (x;y) : @x
0,0:2 1,1:2 2,2:2 3,3:2
* 1,-1 * 1,-1
1,0:2 2,0:2 3,1:2
- 1,1
0,1:2
- 1,1
0,2:2 1,3:2
1,0:2 0,0:2 3,2:2 1,1:2 0,1:2 1,-1 1,1 * 2,2:2 2,3:2 3,3:2 Figure 9: WFT computing derivative and partial derivatives
@f (x;y) : @y
its neighbors. Examples of such WFT are a WFT that simulates the moves of a knight on a 2n 2n chess-board, WFT that implements any type of low or high pass lter (one such lter is shown in Fig. 8) or WFT Di which for any nite resolution computes f (x+h)?f (x) for table step h. Thus WFT Di computes the derivative df (x) in the limit h dx (in nite resolution). WFT Di and similar WFT that compute the partial derivates for functions of two variables are shown in Fig. 9. In Fig. 10 we show the sign of the partial derivatives of Carol. Any positive value is represented by white color, negative by black. Computing of integrals requires to compute linear combinations of values from an unbounded number of neighbors but that can be done, too. WFT shown at the top of Fig. 11, for each resolution, computes h(f (0) + f (h) + f (2h) + . . .+ f (x)) where h is the table step (distance of two neighboring pixels). Thus for the in nite resolution WFT int computes Zx f (x)dx = hlim h(f (0) + f (h) + f (2h) + . . . + f (x)) : !0 0 R R WFT computing (in the limit) 0y 0x f (x; y)dxdy is shown at the bottom of Fig. 11. In all the examples above the same small WFT computes the desired transformation for every resolution and for the in nite resolution as well. For the shifts the limiting case is the identity transformation. However, it is easy to give an average preserving WFT which does not converge to any mapping on in nite resolution. Consider one state 14
@ Carol(x;y) @x
@ Carol(x;y) @ Carol(x;y) @y @x@y Figure 10: The partial derivatives of Carol
WFT with I=F=1, and transitions (0,0:1),(1,0:1), (2,2:1) and (3,2:1).
Theorem 5 Let = f0; 1g and A be a WFA over , Then we can construct WFA B
such that (i) f^B (x) = df^dxA(x) for all x 2 [0; 1] for which df^dxA(x) exists. (ii) f^B (x) = R0x fA (t) dt if the Rieman integral exists.
Proof. Follows from the Examples above and the closure of WFA under WFT [2].
2
Note that (ii) is an extension of a result in [3] where this result was shown for the restricted case of the average preserving WFA. However, if WFA A has n states the construction in [3] yields B with n + 1 states while the application of our two state integrating WFA yields WFA B with 2n states. Theorem 6 Let = f0; 1; 2; 3g and A be a WFA over with f^A : [0; 1]2 ! IR. Then we can construct WFA B such that (i) f^B (x; y) = @@f^iAx@(x;yj y ) for all (x; y) 2 [0; 1]2 for which the derivative exists and all i; j 1. (ii) f^B (x; y) = R0x f^A(t; y) dt for all x 2 [0; 1] for which the Rieman integral exists. (iii) f^B (x; y) = R0y f^A(x; t) dt for all y 2 [0; 1] for which the Rieman integral exists. 15
(iv) f^B (x; y) = R0xR0y f^A (t; s) ds dt for all (x; y) 2 [0; 1]2 for which the Rieman integral exists.
Proof. Follows from the Examples above, the closure of WFA under composition and the closure of WFA under WFT [2].
2
WFT Mallat shown in Fig. 12 computes the coecients of the discrete Haar wavelet transform for any nite resolution and presents them in the Mallat form [8]. It computes the continuous Haar wavelet transform for the in nite resolution. The image Mallat(Carol) is shown in Fig. 13 with the contrast increased. In order to display better all the coecients we compute them unscaled that is use wavelets which are not orthonormal. To get the orthonormal case it is sucient just to adjust the weights of the " transitions of our WFT. All the examples of image transformations here were produced using a menu-driven X-windows based system wftx implemented by P.Rajcani. In wftx images are represented either in pixel form or by WFA. The conversions between these representations are implemented using the WFA inference and the WFA decoding algorithms from [5, 6]. All the operations on images, WFA and WFT, except for the concatenation closure are implemented in wftx. In particular, a WFT can be applied to an image in pixel form (resolution 2n 2n ; n 1) and produce again an image in possibly di erent resolution, or a WFT can be applied to a WFA and produce a WFA. Any of the WFA computing integrals is a typical example of a WFA which has a small number of states but it is highly nondeterministic. For such a WFT, it is much more ecient to apply it to an image in the WFA representation even if it requires converting the image to the WFA representation and then the WFA decoding.
5 Conclusions In the previous section we have shown that many relatively complicated image transformations can be expressed by simple WFT. Since there exists an ecient implementation of WFT, we can use WFT as a powerful design tool for image transformation. In addition, using the wftx system, any WFT can be applied to an image either in the pixel form or in the WFA-compressed form. Due to this property, we can process compressed images without decoding them. It should also be noted that WFT provide a rigorous mathematical description of image transformations. WFA and WFT can also be used with alphabets of di erent size than 22 = 4. Using an alphabet with 32 = 9 letters enables us to manipulate images of size 3n 3n , using 3 2 = 6 letters images of size 3n 2n, e.t.c. 16
If image approximations are sucient, we can process images of arbitrary rectangular size by squeezing/stretching them rst to the closest size 2n 2n and after manipulation unsqueezing/unstretching them back to the original size. This method has been implemented in the wftx system.
17
Z x 0
f (t)dt
:
*
0,0:0:5 1,1:0:5
Z yZ x 0
0
-
1,1
1 4 1 0,1: 4 1 1,0: 4 1 1,1: 4 0,0:
f (x; y )dx dy
0,1:0:5
:
1 4 1 1,1: 4 1 2,2: 4 1 3,3: 4
*
1,1
1 4 1 0,3: 4 1 1,2: 4 1 1,3: 4
1,3:
0,1
1 4 1 0,2: 4 1 2,0: 4 1 1,1: 4 0,0:
K
0,1:
0,3: 2,1: 2,3:
4
1 4 1 1,3: 4 1 3,1: 4 1 3,3: 4
2,2:
Figure 11: WFT computing integrals 18
1 4 1 0,1: 4 1 0,2: 4 1 0,3: 4 1 1,0: 4 1 1,1: 4 1 1,2: 4 1 1,3: 4 1 2,0: 4 1 2,1: 4 1 2,2: 4 1 2,3: 4 1 3,0: 4 1 3,1: 4 1 3,2: 4 1 3,3: 4
0,0:
0,2:
0,3:
0,2:
1,0:0:5 1,1:0:5
1 4 1 2,3: 4 1 3,2: 4 1 3,3: 4
?? @@ @@ ?? @@ ?? @@ ?? ? 1 4 -R@ @@ ?? @@ ?? ? 1 1@ ? 4 @ 4 1 1 ? @@ 4 4 ? 1 @R ? 4 1 1 4 1 2,3: 4 0,1:
0,0:0:5 0,1:0:5
2,2:
0,1
0,0:
0,1
0,1
",0:1
?
* 1,0 ?? @@
@@ ",2:1 @ ",3:1 ? @@ ? ? @ ? @@R 0,0:1 0,0:1 ? 1,1:1 ? 1,1:1 * 0,0 2,2:1 * 0,0 2,2:1 * 0,0 @@ 3,3:1 ? 3,3:1 ? 0,":1 @@ ? 1,":-1 ? @ ? 2,":1 ? 0,":-1 @ 3,":-1 ? 0,":-1 1,":-1 @@ 1,":1 ? 2,":1 2,":1 @@R ? ? ? 3,":1 3,":-1 ",1:1??
0,0:1 1,1:1 2,2:1 3,3:1
0,1
Figure 12: WFT computing Haar wavelet coecients in Mallat form
Figure 13: The image Mallat(Carol) 19
References [1] K. Culik II and S. Dube, Balancing Order and Chaos in Image Generation, Computers and Graphics 17, 4, 465-486 (1993). [2] K. Culik II and I. Fris, Weighted Finite Transducers in Image Processing. Discrete Applied Mathematics 58, 223-237 (1995). [3] K. Culik II and J. Karhumaki, Automata Computing Real Functions, SIAM J. on Computing 23, 4, 789-814 (1994). [4] K. Culik II and J. Kari, Image Compression Using Weighted Finite Automata, Computers and Graphics 17, 3, 305-313 (1993). [5] K. Culik II and J. Kari, Image-Data Compression Using Edge-Optimizing Algorithm for WFA Inference, Journal of Information Processing and Management 30, 6, 829838 (1994). [6] K. Culik II and J. Kari, Ecient Inference Algorithm for Weighted Finite Automata, in Fractal Image Encoding and Compression, ed. Y.Fisher, pp. 243-258, SpringerVerlag (1994). [7] K. Culik II and P. Rajcani, Iterative Weighted Finite Transductions, Acta Informatica, to appear. [8] R. A. DeVore, B. Jawerth and B. J. Lucier, Image Compression through Wavelet Transform Coding, IEEE Transactions of Information Theory 38, 719-746 (1992). [9] S. Eilenberg, Automata, Languages and Machines, Vol. A, Academic Press, New York (1974).
20
