Learning Transformations with Complex-Valued Neurocomputing
International Journal of Organizational and Collective Intelligence, 3(2), 81-116, April-June 2012 81
Learning Transformations with Complex-Valued Neurocomputing Tohru Nitta, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan
ABSTRACT The ability of the 1-n-1 complex-valued neural network to learn 2D affine transformations has been applied to the estimation of optical flows and the generation of fractal images. The complex-valued neural network has the adaptability and the generalization ability as inherent nature. This is the most different point between the ability of the 1-n-1 complex-valued neural network to learn 2D affine transformations and the standard techniques for 2D affine transformations such as the Fourier descriptor. It is important to clarify the properties of complex-valued neural networks in order to accelerate its practical applications more and more. In this paper, first, the generalization ability of the 1-n-1 complex-valued neural network which has learned complicated rotations on a 2D plane is examined experimentally and analytically. Next, the behavior of the 1-n-1 complex-valued neural network that has learned a transformation on the Steiner circles is demonstrated, and the relationship the values of the complex-valued weights after training and a linear transformation related to the Steiner circles is clarified via computer simulations. Furthermore, the relationship the weight values of the 1-n-1 complex-valued neural network learned 2D affine transformations and the learning patterns used is elucidated. These research results make it possible to solve complicated problems more simply and efficiently with 1-n-1 complex-valued neural networks. As a matter of fact, an application of the 1-n-1 type complex-valued neural network to an associative memory is presented. Keywords:
Affine Transformation, Back-Propagation Algorithm, Complex-Valued Neural Network, Generalization Ability, Steiner Circles
INTRODUCTION In recent years there has been a great deal of interest in complex-valued neural networks whose weights, threshold values, and input and output signals are all complex numbers, and their applications such as telecommunications, speech recognition and image processing (Hi-
rose, 2003, 2006, 2013; Nitta, 2008, 2009). The application field of the complex-valued neural network is wider than that of the real-valued neural network because the complex-valued neural network can represent more information (phase and amplitude) than the real-valued neural network (Hara & Hirose, 2004; Buchholz & Bihan, 2008; Mandic, Javidi, Goh,, Kuh &
DOI: 10.4018/joci.2012040103 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Globla is prohibited.
82 International Journal of Organizational and Collective Intelligence, 3(2), 81-116, April-June 2012
Figure 1. Real-valued neuron model. Weights W nm, m = 1, , N and threshold V n are all real numbers. The activation function f is a real function.
Aihara, 2009; Tanaka & Aihara, 2009), and the complex-valued neural network has some inherent properties such as the ability to learn 2D affine transformations (the ability to transform geometric figures), the orthogonal decision boundary (Nitta, 2008), and the property on singular points (Nitta, 2013). In this paper, first, we will examine experimentally and analytically the generalization ability of the 1-n-1 complex-valued neural network which has learned complicated rotations on a 2D plane. Next, we will clarify the properties of the 1-n-1 complex-valued neural network through learning of a transformation on the Steiner circles. As a result, it is shown that two learning patterns are sufficient for the learning ability of 2D affine transformation of a complex-valued neural network. The linear transformation on the Steiner circles is the simplest among linear transformations. That is the reason why we chose the linear transformation on the Steiner circles. Furthermore, we will elucidate the relationship the weight values of the 1-n-1 complex-valued neural network learned a transformation on the Steiner circles or 2D affine transformations and the learning patterns used. As a consequence, it is learned that a complex-valued neural network conducts learning so that the rotation component of the learned complex function is mainly reflected as the sum of the phases of weights. Furthermore, an application of the 1-n-1 type complex-valued neural network to an associative memory is presented where the knowledge obtained in this paper is utilized effectively.
NEURAL NETWORK MODEL A brief overview of neural networks is given. In the early 1940s, the pioneers of the field, McCulloch and Pitts, proposed a computational model based on a simple neuron-like element (McCulloch & Pitts, 1943). Since then, various types of neurons and neural networks have been developed independently of their direct similarity to biological neural networks. They can now be considered as a powerful branch of present science and technology. Neurons are the atoms of neural computation. Out of those simple computational neurons all neural networks are build up. An illustration of a (real-valued) neuron is given in Figure 1. The activity of neuron n is defined as: x = ∑ mW nm X m +V n
(1)
where Wm is the real-valued weight connecting neuron n and m, X m is the real-valued input signal from neuron m , and Vn is the realvalued threshold value of neuron n . Then, the output of the neuron is given by f (x ) . Although several types of activation functions f can be used, the most commonly used are the sigmoidal function and the hyperbolic tangent function. Neural networks can be grouped into two categories: feedforward networks in which graphs have no loops, and recurrent networks where loops occur because of feedback connections. A feedforward type network is made
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
34 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the publisher's webpage: www.igi-global.com/article/learning-transformations-withcomplex-valued-neurocomputing/93078
Related Content Evaluation of the Shopping Path to Distinguish Customers Using a RFID Dataset Takanobu Nakahara and Katsutoshi Yada (2011). International Journal of Organizational and Collective Intelligence (pp. 1-14).
www.irma-international.org/article/evaluation-shopping-path-distinguishcustomers/60751/ Using Swarm Intelligence for Optimization of Parameters in Approximations of Fractional-Order Operators Guido Maione, Antonio Punzi and Kang Li (2013). Swarm Intelligence for Electric and Electronic Engineering (pp. 194-222).
www.irma-international.org/chapter/using-swarm-intelligence-optimizationparameters/72829/ A Clustering Ant Colony Algorithm for the Long-Term Car Pooling Problem Yuhan Guo, Gilles Goncalves and Tienté Hsu (2012). International Journal of Swarm Intelligence Research (pp. 39-62).
www.irma-international.org/article/clustering-ant-colony-algorithmlong/69776/ The Computing of Digital Ecosystems Gerard Briscoe and Philippe De Wilde (2012). Intelligent and Knowledge-Based Computing for Business and Organizational Advancements (pp. 258-275).
www.irma-international.org/chapter/computing-digital-ecosystems/65798/ Design and Optimization of Microwave Circuits and Devices with the Particle Swarm Optimizer Massimo Donelli (2013). Swarm Intelligence for Electric and Electronic Engineering (pp. 1-17).
www.irma-international.org/chapter/design-optimization-microwave-circuitsdevices/72820/
Learning Transformations with Complex-Valued Neurocomputing Tohru Nitta, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan
ABSTRACT The ability of the 1-n-1 complex-valued neural network to learn 2D affine transformations has been applied to the estimation of optical flows and the generation of fractal images. The complex-valued neural network has the adaptability and the generalization ability as inherent nature. This is the most different point between the ability of the 1-n-1 complex-valued neural network to learn 2D affine transformations and the standard techniques for 2D affine transformations such as the Fourier descriptor. It is important to clarify the properties of complex-valued neural networks in order to accelerate its practical applications more and more. In this paper, first, the generalization ability of the 1-n-1 complex-valued neural network which has learned complicated rotations on a 2D plane is examined experimentally and analytically. Next, the behavior of the 1-n-1 complex-valued neural network that has learned a transformation on the Steiner circles is demonstrated, and the relationship the values of the complex-valued weights after training and a linear transformation related to the Steiner circles is clarified via computer simulations. Furthermore, the relationship the weight values of the 1-n-1 complex-valued neural network learned 2D affine transformations and the learning patterns used is elucidated. These research results make it possible to solve complicated problems more simply and efficiently with 1-n-1 complex-valued neural networks. As a matter of fact, an application of the 1-n-1 type complex-valued neural network to an associative memory is presented. Keywords:
Affine Transformation, Back-Propagation Algorithm, Complex-Valued Neural Network, Generalization Ability, Steiner Circles
INTRODUCTION In recent years there has been a great deal of interest in complex-valued neural networks whose weights, threshold values, and input and output signals are all complex numbers, and their applications such as telecommunications, speech recognition and image processing (Hi-
rose, 2003, 2006, 2013; Nitta, 2008, 2009). The application field of the complex-valued neural network is wider than that of the real-valued neural network because the complex-valued neural network can represent more information (phase and amplitude) than the real-valued neural network (Hara & Hirose, 2004; Buchholz & Bihan, 2008; Mandic, Javidi, Goh,, Kuh &
DOI: 10.4018/joci.2012040103 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Globla is prohibited.
82 International Journal of Organizational and Collective Intelligence, 3(2), 81-116, April-June 2012
Figure 1. Real-valued neuron model. Weights W nm, m = 1, , N and threshold V n are all real numbers. The activation function f is a real function.
Aihara, 2009; Tanaka & Aihara, 2009), and the complex-valued neural network has some inherent properties such as the ability to learn 2D affine transformations (the ability to transform geometric figures), the orthogonal decision boundary (Nitta, 2008), and the property on singular points (Nitta, 2013). In this paper, first, we will examine experimentally and analytically the generalization ability of the 1-n-1 complex-valued neural network which has learned complicated rotations on a 2D plane. Next, we will clarify the properties of the 1-n-1 complex-valued neural network through learning of a transformation on the Steiner circles. As a result, it is shown that two learning patterns are sufficient for the learning ability of 2D affine transformation of a complex-valued neural network. The linear transformation on the Steiner circles is the simplest among linear transformations. That is the reason why we chose the linear transformation on the Steiner circles. Furthermore, we will elucidate the relationship the weight values of the 1-n-1 complex-valued neural network learned a transformation on the Steiner circles or 2D affine transformations and the learning patterns used. As a consequence, it is learned that a complex-valued neural network conducts learning so that the rotation component of the learned complex function is mainly reflected as the sum of the phases of weights. Furthermore, an application of the 1-n-1 type complex-valued neural network to an associative memory is presented where the knowledge obtained in this paper is utilized effectively.
NEURAL NETWORK MODEL A brief overview of neural networks is given. In the early 1940s, the pioneers of the field, McCulloch and Pitts, proposed a computational model based on a simple neuron-like element (McCulloch & Pitts, 1943). Since then, various types of neurons and neural networks have been developed independently of their direct similarity to biological neural networks. They can now be considered as a powerful branch of present science and technology. Neurons are the atoms of neural computation. Out of those simple computational neurons all neural networks are build up. An illustration of a (real-valued) neuron is given in Figure 1. The activity of neuron n is defined as: x = ∑ mW nm X m +V n
(1)
where Wm is the real-valued weight connecting neuron n and m, X m is the real-valued input signal from neuron m , and Vn is the realvalued threshold value of neuron n . Then, the output of the neuron is given by f (x ) . Although several types of activation functions f can be used, the most commonly used are the sigmoidal function and the hyperbolic tangent function. Neural networks can be grouped into two categories: feedforward networks in which graphs have no loops, and recurrent networks where loops occur because of feedback connections. A feedforward type network is made
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
34 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the publisher's webpage: www.igi-global.com/article/learning-transformations-withcomplex-valued-neurocomputing/93078
Related Content Evaluation of the Shopping Path to Distinguish Customers Using a RFID Dataset Takanobu Nakahara and Katsutoshi Yada (2011). International Journal of Organizational and Collective Intelligence (pp. 1-14).
www.irma-international.org/article/evaluation-shopping-path-distinguishcustomers/60751/ Using Swarm Intelligence for Optimization of Parameters in Approximations of Fractional-Order Operators Guido Maione, Antonio Punzi and Kang Li (2013). Swarm Intelligence for Electric and Electronic Engineering (pp. 194-222).
www.irma-international.org/chapter/using-swarm-intelligence-optimizationparameters/72829/ A Clustering Ant Colony Algorithm for the Long-Term Car Pooling Problem Yuhan Guo, Gilles Goncalves and Tienté Hsu (2012). International Journal of Swarm Intelligence Research (pp. 39-62).
www.irma-international.org/article/clustering-ant-colony-algorithmlong/69776/ The Computing of Digital Ecosystems Gerard Briscoe and Philippe De Wilde (2012). Intelligent and Knowledge-Based Computing for Business and Organizational Advancements (pp. 258-275).
www.irma-international.org/chapter/computing-digital-ecosystems/65798/ Design and Optimization of Microwave Circuits and Devices with the Particle Swarm Optimizer Massimo Donelli (2013). Swarm Intelligence for Electric and Electronic Engineering (pp. 1-17).
www.irma-international.org/chapter/design-optimization-microwave-circuitsdevices/72820/
