Optical Learning Chip Jun Ohta, Yoshikazu Nitta and Kazuo Kyuma Optoelectronics Mitsusbishi Laboratory, RWCP 8-1-1 Tsukaguchi-Honmachi, Amagasaki, Hyogo, 661, JAPAN phone: 06-497-7080, fax: 06-497-7288, e-mail:
[email protected]
ABSTRACT A novel type of an optical neurochip with learning capability and memory function is reported. The neurochip is a three dimensional optoelectronic integrated circuit consisting of a light emitting diode array and a variable sensitivity photodetector (VSPD) array. The principle of operation and the fundamental characteristics are described. By using the fabricated optical neurochip with 32 neurons and 32 32 synapses, experiments of on-chip learning based on the backpropagation and Boltzmann machine learning algorithms have successfully been demonstrated.
INTRODUCTION Optoelectronics is expected to play an important role in hardware implementation of neural networks because of its innate parallelism, high-density interconnection, and direct image processing abilities [1]. This summary describes an optical learning chip with variable synaptic interconnections developed in our laboratory. This chip enables to perform on-chip learning by using an internal analog memory function. First, we will describe the structure and the operation principle of the chip. Next a matrix addressing scheme using such a memory effect is introduced for realizing an on-chip learning and a large scale integration. An optical neurochip with 32 neurons has been fabricated and successfully used for the experiments of learning based on the back-propagation (BP) [2] and Boltzmann machine learning algorithms [3].
MONOLITHIC OPTICAL NEUROCHIP Figure 1 shows the schematic diagram of the monolithically integrated optical neurochip and its fundamental element. It consists of a line-shaped LED array and a 2-D VSPD array. This chip performs an optical vector-matrix multiplication in parallel. 32-neuron chip has been fabricated, and 64-neuron chip is under developing. The size of each element was 6060 m2 and the chip area was 6.36.3 m2 . The processing speed is over 10MHz. The VSPD, which has the combined functions of a spatial light modulator (SLM) and a photodetector (PD), was employed as the synaptic device. The principle of operation of the VSPD is that the sensitivity of the PD Figure 1: Schematic diagram of optical neuis varied by the externally applied voltage. We have employed a rochip. metal-semiconductor-metal (MSM) structure as the VSPD. MSMVSPD has analog memory function which is based on the phenomenon that information (i.e. an optical power and/or an electrical voltage) is stored in the VSPD as the sensitivity when the light and the voltage are applied to the VSPD at the same time. Both positive and negative values can be stored by changing the polarity of the applied voltage. The storing time is about 20 minutes.
MATRIX ADDRESSING OF 2-D ARRAY By using the analog memory function, we have proposed a novel type of a matrix addressing scheme which enables to change the sensitivity (synaptic weight) of all the VSPD elements in the 2-D array [2]. The most outstanding feature of this scheme is that the 2-D VSPD array has internal memory, thus no external memory is needed. The principle of operation for the matrix addressing is illustrated in Fig. 2. In the write (learning) phase, at first, the current is injected into the LED elements in the first row so that all the LEDs in this row are turned on and emit the same optical power towards the associated VSPD elements. Simultaneously, the vector of write voltages, which corresponds to the signals (synaptic weights) to be stored in the first row, are applied to the VSPD elements in parallel. In this manner, the write This PostScript version was created from the original authors’ English article by the Japanese Information Sciences Project (JISP), at New York University, in collaboration with the RWCP, aiming at worldwide access to the information. Every precaution has been taken to avoid errors arising from the conversion of printed documents to electronic form, however, should there be any discrepancies, the JISP bears sole responsibility for them. Email address:
[email protected], URL http://jisp.cs.nyu.edu/.
? 137 ?
signals are memorized in the VSPD elements of the first row as the sensitivity. The same procedure is sequentially repeated from the first row to the last row in one frame scan. Thus the sensitivity of the 2-D VSPD array is memorized in the form of matrix. In the read-out (retrieval) phase, the read-out signals (states of neurons) are injected into th e LED elements of all the rows in parallel. And the photocurrents (product of neuron states and synaptic weights) which is proportional to the vector-matrix product, are obtained from the output electrodes of the VSPD array.
LEARNING PROCESS USING OPTICAL NEUROCHIP The matrix addressing scheme is used to implement learning algorithm [2]. The voltage corresponding to the update signal of the synapse is applied to the Figure 2: Schematic illustration of VSPD electrodes, while simultaneously illuminating the VSPD array with the matrix addressing scheme. uniform light from the LED array. Increment and decrement of the synaptic weight are realized only by changing the polarity of the applied voltage. The learning experiments based on the BP and Boltzmann learning algorithms have been performed [2], [3]. The experimental setup for BP is shown in Fig. 3 [2]. To implement the three layered network, the chip was spatially divided into two regions, that is, the input-to-hidden (1,2) and hidden-to-output (2,3) regions. The number of the neurons in these layers are 8, 8, and 3, respectively. Total numbers of the LED and VSPD elements for implementing the network were, respectively, 16 and 88. The chip is used to train a set of 14 patterns with 8-bit codes into 3 categories through the BP learnFigure 4: Learning curve Figure 3: Experimental setup ing algorithm. To correct the output signals, for BP the synaptic weights were modified according to the BP algorithm by using the matrix addressing scheme. The experimental results are shown in Fig. 4. It is shown that the input patterns were correctly classified into the 3 categories after about 120 presentations. The computer-simulated results are also shown in the figure and it is found that they relatively agree with the experimental ones, where the same parameters were used in the simulation. Boltzmann machine learning algorithm is also implemented. The experimental setup for the learning is shown in Fig. 5 [3]. Instead of the feedforward network as shown in Fig. 3, the selforganization system can be built by introducing feedback circuits in the chip. The stochastic process was introduced by adding a Gaussian distributionnoise to the threshold value . The learning and antilearning phases were performed alternatively. In each phase, the thresholded output signal from the chip is fed-back to the VSPD applied voltage, while the input LED is still turning on. The Hebbian and anti-Hebbian rules were automatically accomplished only Figure 5: Experimental setup for Boltzmann by changing the polarity of the applied voltage to VSPD. Repeated machine learning. this process, the synapse weights were updated and self-organized. We have successfully demonstrated the learning of XOR process by this learning system. The neuron number of input, hidden, and output layers were 2, 1, and 1, respectively.
REFERENCES [1]
K. Kyuma and S. Tai, IEICE Trans. Electron., E76-C, 1070, 1993.
[2]
Y. Nitta, J. Ohta, S. Tai and K. Kyuma, Appl. Opt., 32, 1265, 1993.
[3]
Y. Nitta, J. Ohta, S. Tai, and K. Kyuma, Proc. 53rd Jpn Appl. Phys. Spring Meeting, 31a-ZC-7, 1993.
? 138 ?
