2010 International Conference on Pattern Recognition
A Geometric Radial Basis Function Network for Robot Perception and Action E. V´azquez-Santacruz and E. Bayro-Corrochano CINVESTAV Unidad Guadalajara, Av. Cient´ıfica 1145, El Baj´ıo, Zapopan, Jalisco, M´exico evazquez,
[email protected]
Abstract
learn quickly. This kind of networks is useful in solving problems where the input data are corrupted with additive noise. The transformation functions used in hidden layer are based on a Gaussian distribution.
This paper presents a new hypercomplex valued Radial Basis Network. This network constitutes a generalization of the standard real valued RBF. This geometric RBF can be used in real time to estimate changes in linear transformations between sets of geometric entities. Experiments using stereo image sequences validate this proposal.
1
2
The geometric algebra give us the possibility to describe in a compact way to the lines, the planes, and other geometric entities, as well as the representation of rotations, translations and other transformations. We are using this mathematical system in order to update the weights of the Geometric RBF Network (GRBF-N). In geometric algebra [2] we define the Clifford product as the sum of the inner product and the wedge product of two vectors: ab = a · b + a ∧ b. The Clifford product is lineal, associative and anticommutative.
Introduction
The Complex-valued Neural Networks have been studied in the cases of single neuron, multilayered perceptrons and recurrent networks [4]. Hypercomplex numbers, i.e., higher dimensional number system than complex value are also used to model neural networks. Quaternion is a four-dimensional hypercomplex number system and there are several applications of these numbers to neural networks. The MLPs with quaternionic neurons and their learning algorithms have been proposed [6] [1] [7] [2]. Even, there is a geometric neural network [3] which is based on multilayer perceptron and uses a genetic algorithm for training. Nevertheless, there are few investigations related to hypercomplex neural networks where dimension is higher than four, using RBF Networks (RBF-N) [5]. From the viewpoint of computational applications, it is important to define a network model that let us to use data expressed in hypercomplex numbers. This can be used to find transformations between entities expressed in four dimensions or higher, and it can be done using the geometric algebra framework. In this work, we create a model where inputs, outputs, weights and hidden neurons are defined as hypercomplex numbers. We obtain the existing transformation between input and output through a combination of geometric entities which allows a more natural description than others. We are using a RBF-N because it is good in modeling nonlinear data and can 1051-4651/10 $26.00 © 2010 IEEE DOI 10.1109/ICPR.2010.725
Geometric Algebra
2.1
Geometric Algebra Gp,q,r
A geometric algebra is a lineal space of dimension 2n , n = p + q + r, and (p, q, r, ∈ N ). In a geometric algebra the notation Gp,q,r represents the p vector basis that squares to +1, the q basis that squares to -1 and the r vector basis that squares to 0. So the 3D space (G3,0,0 ) has the following blade basis:
1 , e1 , e2 , e3 , e23 , e31 , e12 , e123 ≡ I |{z} | {z } | {z } | {z } scalar
vectors
bivectors
(1)
trivectors
In the equation (1) the scalar is a blade1 of zero degree, the vectors are of first degree, the bivectors are blades of second degree and the pseudoscalar I is a blade of third degree. I is used to convert a geometric entity from its standard form to its dual form and vice versa. A multivector is a linear combination of blades of different degrees. Let Ar and Bs be multivectors of degree r and s respectively; the Clifford product 1 The
2953 2965 2961
elements that limit a geometric algebra are called blades.
of them is expressed as: Ar Bs
=
hAr Bs ir+s + hAr Bs ir+s−2 + ... + hAr Bs i|r−s| .
(2)
From (2) the general definitions for the inner and wedge products of multivectors can be obtained: Ar · Bs = hAr Bs i|r−s| .
(3)
Ar ∧ Bs = hAr Bs ir+s .
(4) (a)
2.2
Properties of the Multivectors
Figure 1. RBF Networks
Pr With a multivector of degree r, Ar = i=0 hAr ii is possible to carry out the following operations: Clifford conjuer ) and Involution degree (A br ). gation (Ar ), Reversion (A Ar
er A
br A
=
= =
r X
i=0 r X
i=0 r X
(−1)
(−1)
i(i+1) 2
i(i−1) 2
hAr ii
(5)
hAr ii
(6)
(−1)i hAr ii
3
(7)
The dual of a multivector A in Gn 2 . is described by: A∗ = AI−1 n . The dual of a r-blade is a (n-r)-blade. The exponential of a multivector A can be expressed as: ∞ X Ak k!
where k(·) is a strictly positive radially symmetric function (kernel) with a unique maximum at its ‘centre’ cj and which drops off rapidly to zero away from the centre, and has an appreciable value only when the distance ||X − cj || is smaller than σj . The parameter σj is the width of the receptive field in the input space from unit j. Given an input vector X, the output of RBF-N is the L-dimensional activity vector Y , whose lth component (l = 1, 2, . . . L) is given by,
(8)
k=0
2.3
Rotors in G3
Yl (X) =
The rotors are used to rotate geometric objects. A rotor in e = 1, where: G3 satisfies the relation RR R = u0 + u1 e23 + u2 e31 + u3 e12 |{z} | {z } scalar
scalar
(10)
4
(11)
where n is a unit bivector that represents the rotation plane and θ ∈ R represents the rotation angle. 2 To
wlj zj (X)
(13)
The accuracy of these networks is controlled by three parameters: number of basis functions, their location and their width. These values and weights applied to the RBF function outputs are determined by the training process. K-Means clustering is one of many methods to find the cluster centers which are used as centers for the RBF functions. To find the weights of output layer is convenient to use a gradient-descent method.
(9)
bivectors
The rotor can be expressed in exponential form as: θ θ θ R = exp − n = cos − sen n 2 2 2
M X
m=1
bivectors
e = u0 − u1 e23 − u2 e31 − u3 e12 R {z } |{z} |
A Real Valued RBF Network
A RBF-N has a feed forward structure consisting of a single hidden layer of m locally tuned units, which are fully interconnected to an output layer of L linear units using weights. All hidden units simultaneously receive the n-dimensional real valued input vector X (Figure 1(a)). Each hidden-unit output zj is obtained by closeness of the input X to an ndimensional parameter vector cj associated with the jth hidden unit. The response of the jth hidden unit (j = 1, 2, ..., m) is assumed as, ||X − cj || zj = k (12) σ2 j
i=0
exp(A) =
(b)
Geometric Networks
Radial
Basis
Function
Let us to define our scheme as g(P, X) where P is the set of parameters of our network, X is the input and g(·) is the network (figure 1(b)); which is defined by hypercomplex
represent the notation G3,0,0 ≃ G3 and G4,1,0 ≃ G4,1 .
2962 2966 2954
valued neurons. By adaptive learning, P is tuned so that the training data fits the network model as well as possible. g(·) uses Gaussian basis functions to define the closeness of X ˆ as weights of to the centers ˆ c. Our scheme uses rotors (R) output layer which are combined linearly to define the output ˆ ˆ Y of the network. X, Y , R, c are in G3 . These parameters and σi values of each center define P .
4.1
where zi (n) ∈ R and Ri (n), ci (n), Xi (n), Yi (n) ∈ G3 . As we can observe, the GRBF-N uses the Clifford proP f duct. Note that g(P, X, n) = M i=1 Ri (zi (n)ci (n))Ri
5
Experimental Analysis
Training of GRBF Networks
The training set is a labelled par Xi , Yi that represents associations of a given mapping. Given a number of centers, the adaptive learning process starts when the first training input pair is presented to the network. GRBF-N uses the K-Means clustering algorithm to define the centers of each Gaussian basis function and a Euclidean distance to indicate the nearness of Xi to each center ci . The parameter σi is defined for each Gaussian unit to equal the maximum disˆ that best approximates tance between centers. To determine R the sample data Xi , GRFB-N uses an ordinary least squares ˆ considering to romethod (LMS). Our scheme update the R tate the center ci using the associated Ri and the respective radial basis function output. As we can see, K-Means clustering algorithm and LMS algorithm proceed with their own individual computations in a concurrent way, which accelerates the training process. Our GRBF-NN works iteratively as follows: 1) Initialization. Choose random values for initial centers ck (0) 2) Iterate until no noticeable changes are observed in the centers ck or there is no more training input pair a) Sampling. Draw a sample input x coded by a hypercomplex number from the input space I. x is input into the algorithm at iteration n b) Similarity definition. k(x) denote the index of the best-matching center for input x. Find k(x) at iteration n by using the minimum-distance Euclidean criterion: k(x) = argmin||x(n) − ck (n)||, k = 1, 2..., m, k
ck (n) is the center of the kth radial function at iteration n. Note that the dimension of the k-vector x is the same as that of the k-vector ck and in this case is 3. c) U pdating. Adjust the centers of the radial basis functions: ck (n + 1) =
(
ck (n) + ρ[x(n) − ck (n)], if k = k(x), ck (n), otherwise
ρ is a learning-rate parameter in the range 0 < ρ < 1
d) Error. e(n) = Y (n) −
Pm
i=1
fi (n) Ri (n)(zi (n)ci (n))R
e) Rotor vector updating.
Ri (n + 1) = Ri (n) + ηzi (n)e(n) η is a learning-rate parameter in the range 0 < η < 1
f) Continuation. Increment n by 1
In this approach we use data available from stereo camera. We have objects whose axis of rotation is marked by two colored well-distributed points. With vision techniques, these two points are identified in 3D space and then the orientation of the object is obtained. Let I be an orientation of an object and O the orientation of another object (or the same) that are expressed using four dimensional multivectors. Then we construct input pairs (for training and testing) only changing a little I to define Xi and O for Yi (figure 4). We use these data sets to train and test the GRBF-N. Figure 2 (above) shows that the rotors values converge to the expected value and this is also observed in figure 2 (below) where is showed that GRBF-N outputs are very close to the desired values. In these experiments we are using four dimensional hypercomplex numbers to code orientations of objects and rotors. We use η = 0.1 ρ = 0.3, three centroids, one input and one output. Figure 3 shows how our network estimates the transformation needed to achieve the target orientation from an initial one, in these sub-figures we can appreciate both orientations. At first, the output of the network is far from the desired orientation. As the network learns, its output is closer to the desired goal. Also, our scheme converges so that using diferent instances of testing data we get an error of 0.039 which is defined using an average of the Euclidean distance between target outputs and GRBF-N outputs. This indicates that our scheme works well. Figure 4 illustrates that GRBF-N is useful to define a more complex movement constructing an array of blocks which can learn a specific transformation in such way that GRBF-N i learn the transformation i. Geometric algebra helps here to define a geometric entity as circles, spheres, planes (see section 2) so that we can have transformations between orientations of different type of geometric entities using multivectors. This can be used to teach a robot what the needed transformations are, in order to move its arms or legs. Using a powerful computing equipment and external cameras for the robot we can achieve this structure and the robot will know in real time which transformation apply to move a part of its arm or leg. Also, the robot could move an object to another orientation using its arms.
2963 2967 2955
not require further processing. The adaptive learning of this scheme is very important to apply it in real environments. It can be seen how the centers are self-organized but only once we have defined a number of hidden units. The self organized network remains for our future work. It is important to explore more applications of GRBFN such as learning of more complex movements as we mentioned.
0.8 r i j k
0.6
0.4
Rotor weights
0.2
0
-0.2
-0.4
-0.6
-0.8 0
50
100 Iteration
150
200
1.4 r i j k
1.2 1 0.8
Output error
0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 0
50
100 Iteration
150
200
Figure 2. Rotors vector evolution (above). Output
Figure 4. Different types of transformations learned
error during learning process (below).
by our GRBF-N
References [1] P. Arena, L. Fortuna, G. Muscato, and M. G. Xibilia. Multilayer perceptrons to approximate quaternion valued functions. Neural Netw., 10(2):335–342, 1997. [2] E. Bayro Corrochano. Geometric Computing for Perception Action Systems: Concepts, Algorithms, and Scientific Applications. Springer, 2001. [3] E. Bayro-Corrochano, R. Vallejo, and N. Arana-Daniel. Geometric preprocessing, geometric feedforward neural networks and clifford support vector machines for visual learning. Neurocomput, 67:54–105, 2005. [4] A. Hirose. Complex-Valued Neural Networks. Springer, 2006. [5] X. L. Mingyu Li *, Songbai He. Complex radial basis function networks trained by qr-decomposition recursive least square algorithms applied in behavioral modeling of nonlinear power amplifiers. Medical Physics, 32:2371– 2379, 2005. [6] H. K. F. P. N. Matsui1, T. Isokawa1 and H. Nishimura. Quaternion neural network with geometrical operators. Journal of Intelligent and Fuzzy Systems, 15(3-4), 2004. [7] N. K. N. M. Teijiro Isokawa, Haruhiko Nishimura. Fundamental properties of quaternionic hopfield neural network. In Neural Networks, 2006. IJCNN ’06. International Joint Conference on, volume 1, 2006.
Figure 3. GRBF output during learning process.
6
Conclusions
We have designed a new network using hypercomplex numbers to define its parameters. The main advantage of using rotors over other hypercomplex numbers is that rotors may be defined in any dimension and a rotor can also rotate blades of any grade. That is, not only vectors but also lines, planes, and any other geometric objects that can be represented by a blade may be rotated with a rotor. The GRBF-N lets to know the existing transformation between the k-vectors at the input and output through a combination of geometric entities which allows a more natural description of the involded transformation. We applied our scheme to a real problem where we get input data from stereo camera. We have shown that GRFB-N can give us the needed transformation in rotors which represents an advantage because this information is immediately useful, it does 2964 2968 2956
