Mar 26, 1995 - cells; this is called clonal selection. The selected B-cell proliferates and secretes an antibody equipped with the same receptors as those on its ...
Immune Pattern Recognition System Toru Ohira
TR-95-006
March 26, 1995
Sony Computer Science Laboratory Inc. 3-14-13 Higashi-gotanda, Shinagawa-ku, Tokyo, 141 JAPAN
Copyright
c
1995 Sony Computer Science Laboratory Inc.
In the proceedings of International Workshops on Biologically Inspired Evolutionary Systems (BIES*95), Tokyo, Japan, 1995
Immune Pattern Recognition System Toru Ohira March 26, 1995
1
Abstract
A model of the immune system which performs a pattern recognition task is presented. We design and construct the model by incorporating the mechanisms of the natural immune system. Experiments are performed to investigate the eciency of the model in multiple pattern (antigen) recognition. This model is extended to an evolutional model in which individuals with the immune system adapt to a given environment which consists of various antigens. The eciency of response of the model immune system of such adapted individuals is shown to be closely related to the structure existing behind its environment.
2
Introduction
Biological systems have provided us a resource of ideas and principles in constructing information processing systems. Modelings of the brain have led to various neural network algorithms. Another example is the genetic algorithm (GA) which has derived its designing principles from evolution. These information processing systems have been found successful in various engineering applications and are topics actively being researched. In this paper, we take the immune system, which is a defense system of the body against foreign cells and molecules, as a biological system to model after. Many theoretical investigations and modeling of the immune system have taken the approach of mapping into coupled nonlinear dynamical systems and solving di erential equations of motion of corresponding parameters[1{ 4]. For engineering application, an arti cial immune system for computers against computer viruses has been constructed [5]. There are several e orts seeking correspondence between GA and the immune system, however, research in this direction has been rather limited[6{10]. We build on previous and rather recent attempts at modeling the immune system to construct an information processing system [6,7]. In these works, the fact that the immune system uses similar operations to those found in evolution such as crossover and mutation is noted and utilized in constructing models within the framework of GA. Our basic hypothesis is that by re ecting more of the functionality of the natural immune system, we can improve aspects of the model. With this in mind, we present a model which is closer to the actual functioning of the immune system. Investigations on the pattern recognition eciency of this model have revealed improvements over the previously proposed system. We then extend the model to an evolutionary adaptive model in a given environment. The performance of the evolution model is shown to be related to the structures of the environment to which it has adapted.
3
A Brief Overview of the Immune System
The immune system is composed of various interactions of di erent types of cells and chemical factors with much of its mechanism remaining to be unveiled. Here, we give a brief and simpli ed account of what is known of the system. Its most characteristic feature is its antigen speci c response. This pattern recognition is performed by the binding of "receptors" on the surface 1
4. Standard GA Model
2
of two lymphocytes (B-cell and T-cell), and on antibodies secreted by corresponding B-cells. The recognition of an antigen by receptors on the lymphocytes leads to the activation of these cells; this is called clonal selection. The selected B-cell proliferates and secretes an antibody equipped with the same receptors as those on its surface. The T-cell regulates the activity of immune responses by producing a variety of factors called interleukins which either accelerate or suppress the production of antibodies. Binding of the antibody to the antigen leads to destruction of the infected cells. The immune system is equipped with the function of memory, and a speci c antigen which activated it in the past is "learned" by the system so that it e ectively responds to another invasion by the same antigen. Other elements are also involved in the immune response, however, we limit ourselves for our modeling of the system here to the above{mentioned function of the immune system. One central issue in investigating this system is how it prepares for the almost limitless diversity of antigen patterns. Currently, this diversity is thought to be a consequence of two processes: recombination of gene segments and somatic mutation. The genes responsible for the variable part of the receptors are kept in segmented form and the diversity to prepare for the variety of antigens is achieved by the process of their recombination. Once the clonal selection is made, a very high rate of mutation occurs in the process of proliferation of the B-cell. The diversity obtained by these two processes is estimated to be around 1015.
4
Standard GA Model
As a starting point to construct a model of the immune system, let us look at a previously proposed GA model system by Forrest et al.[6]. The core of the immune reaction is the interaction between antigen and receptors on antibodies. This process is called an "antigen-antibody reaction" and the reaction is modeled by expressing both antibody and antigen by a xed length of binary strings proposed by Farmer et al.[8]. The anity between an antibody and an antigen is given as the sum of XOR operation between two bits at the same location on each of them, for example, antibody 101011100101110001 antigen 011100101110110010 anity 110111001011000011 = 10 The higher anity of an antibody with a given antigen indicates a higher chance of clonal selection on that antibody. Forrest et al. have considered the problem of developing antibodies which match a given set of antigens using this formalization and the standard GA. ( The recombination of gene segments in the immune system is believed to take place before the clonal selection to create a diversity of antibodies. Hence, as noted in their work as well, the use of crossover does not have direct correspondence in the process of proliferation after clonal selection.) This is a problem of multipoint search or multipattern recognition. They investigated questions relating to the multipattern recognition with this GA model such as the recognition capacity. We build on this GA immune model and focus on the issue of the eciency of response by the model systems: how fast can the system create antibodies to a presented set of antigens?
5
Models
The emphasis of Forrest's model is on how the GA model can deal with problems posed to the immune system. From the point of view of modeling of the system, the standard GA model is not faithful, as mentioned with respect to the use of crossover. Here, we would like to construct and investigate models which re ect and emphasize the functioning of the natural immune system.
5. Models
3
5.1 A model with emphasis on clonal selection and somatic mutation We discuss a modi ed model which incorporates the clonal selection and somatic mutation. The rst change is to take out the operation of the crossover, which does not have a direct correspondence at the stage of proliferation of selected antibodies; the model thus emphasizes somatic mutation as the main procedure at that stage. The second modi cation emphasizes the clonal selection. Given the initial set of antibodies, a new set is created from the highly tted antibodies in the initial set. Somatic mutation takes place in this process. In generating the next set of antibodies, we keep the antibody with highest tness function as "elite" without mutation, while at the same time creating a group of "semi-elite" antibodies, which di ers from the "elite" by just a single bit. For other antibodies we put on the normal probabilistic selection and mutation. This modi cation selectively provides a high mutation rate to the "elite" and creates a "semi-elite" group. (We maintain the size of the group including the "elite" at 10 percent of the entire antibody population in the system.) The motivation for this modi cation is the interpretation of the somatic mutation after the clonal selection in the biological immune system as a means of exploring in the vicinity of the best matched antibody to seek an even better match. (We note that the model is designed so that "elite" here is the highest anity antibody on a statistical average.) We investigated the eciency of creating antibodies in a case of models with N=100 antibodies for m = 3 antigens. The length of antibodies and antigens are set as L = 64. The three antigens are taken to be 1111111111....111 0000000000....000 1010101010....010 Initial sets of N=100 antibodies are created "randomly", i.e., each bit of the string is chosen to be 1 or 0 with equal probability of 0.5. The procedure for the experiment is as follows. We randomly pick one antigen and several (0:15 2 N ) antibodies at each iteration and the antibody with highest anity has its tness value increased by the value of anity. We repeat this process to obtain tness values for all the antibodies. Using the tness function, we evolve the antibody using GA up to G = 400 time steps. This will be repeated for T R = 30 trials; we then take an average of the number of "completely matching antibodies"(or "100 percent correct antibodies") which have an anity = L = 64. We compare the three models: the standard GA model with two-point crossover and mutation with crossover rate C = 0:6, mutation rate M = 0:001; the model with mutation only with M = 0:001; and the model with the "semi-elite" group with mutation rate for non-"semi-elites" M = 0:001. Figure 1 indicates the average number of completely matching antibodies in the system at each time step. As the graph indicates, the best performance is shown by the model with the "semi-elite" group. We note that over a longer time the number of completely matching antibodies is slightly lower with the model re ecting the existence of the "semi-elite" group. One may ask that the model with the "semi-elite" group may create antibodies strongly for a single antigen, and that the better performance with respect to the eciency of response is the result of sacri cing the system's capacity of recognition (i.e., number of antigens to which the system can develop completely matching antibodies simultaneously). As discussed elsewhere [11], this is not the case; the model with the "semi-elite" group shows an increase in recognition capacity over the standard GA model or the model with mutation only. We next consider the problem of how the size of the "semi-elite" group a ects the eciency. We carried out an experiment with the above settings of N=100 and m = 3 to vary the size of the "semi-elite" group. Some of the results are shown in Fig. 2. We found that in this example improvement in eciency is maximum when the size of the "semielite" exceeds about 35. Though not shown here, we also observed that with the size exceeding
4
5. Models
50
Average 40 number of antibodies 30 per antigen 20
(a)
(b)
(c)
10 0
50
100
150
200
250
300
350
Time S teps
Figure 1: Comparison of eciency of response of the models: (a) mutation and crossover (b) mutation only (c) "semi-elite" model. 50 40
Average number of antibodies 30 per antigen
(b) (c) (d)
20
(a) 10 0
25
50
75
100
125
150
175
Time S teps
Figure 2: Comparison of eciency of response of the models by varying "semi-elite" group size: (a)5, (b)10, (c)25, (d)40. 60, the response gets slower. More thorough investigation varying N and m is needed, however, we can conjecture from this result the existence of an optimal "semi-elite" group size most ecient for the response of the model system. Our results indicate that the model which re ects and emphasizes the clonal selection and somatic mutation of the immune system performs better with respect to eciency of pattern recognition. From this result, in reverse, we can hypothesize that one role of somatic mutation is to enhance the eciency in the natural immune system, which standard GA has not taken advantage of. Further investigation is required to determine the nature of improvement with respect to various parameter values.
5.2 Model re ecting adaptation to its environment We here construct an extended model in which an individual evolves its antibody creation structure to adapt to the environment. The main motivation for constructing this model is a hypothesis that the immune system placed in a particular environment has evolved so that it has adapted to the distribution of antigens in that environment, even though we do not yet know exactly how the natural immune system does so. The model proposed here incorporates the gene library model by Hightower et al.[7] and the model with the "semi-elite" group. It consists of three parts, each of which is described in the following. The question we ask with respect to this model is whether the structure behind an environment a ects the eciency of the response.
Environment We create N A = 20 antigens, each with a length L = 64 bits, and call this set of antigens an environment. To examine the response performance, we prepare two environments: random and structured. The random environment consists of antigens created randomly, and the structured environment consists of antigens which are created with rules. The rule we employ to create the structured environment is as follows. We number the N A antigens and
6. Summary and Conclusion
5
set the rst bit in each antigen to be 1. The j th antigen has bit number 1 + j 2 m (m=1,2, 3, ...) to be 1 with other bits to be 0. Concretely, 1111111111111111111111111...... 1010101010101010101010101.... 1001001001001001001001001.... 1000100010001000100010001.... etc.
Evolution Evolution of the individual uses a mechanism similar to that of Hightower et al.[7]
using GA. We create N I = 100 individuals. The chromosome of each individual has four compartments each of which consists of 8 gene parts of 16 bit length. Hence, an individual is represented by a binary string of 512 bits. We evolve these individuals with GA. The tness function is calculated using S = 10 sample antibodies of 64 bit length which each individual can create by combining 4 parts picked from each compartment. This set of antibodies is presented with one antigen per loop and the antigen \score" is the highest anity. The tness function of an individual is the average value of the \scores of the antigens", which is created by this process for each individual. Using this tness function we evolve the individuals using GA. GA employed here uses the elite strategy with mutation rate of M = 0:001 and 2-point crossover with rate C = 0:6. The set of individuals is evolved to 1000 generations and we pick the best individual at that stage. In this manner, an adapted individual is created for each of the two environments.
Antibody Generation We randomly pick some antigens from the environment and present
them to two individuals: one has evolved its gene parts to adapt to the environment as described above, the other has random gene parts (i.e., unevolved). Each individual generates an initial set of 100 antibodies and uses a model with the "semi-elite" group in the previous section. We x the number of antigens to be picked as 4 and 8 and compare the eciency of response between the two individuals for each of the two environments (Fig. 3 and Fig. 4). As these results indicate, there are no notable di erences between the two individuals in the "random" environment. However, in the "structured" environment, the evolved individual is more ecient in response to the antigens than the non-evolved one. This suggests that the evolution considered in this model has more meaning in the "structured" environment. The actual situation re ecting the environment which the biological immune system deals with is believed to be somewhere between the random and the structured. The relation between the environment and evolution in this model requires further study.
6
Summary and Conclusion
We proposed here a model of the immune system which, by creating a "semi-elite" group places more emphasis on clonal selection and somatic mutation of the natural system than the standard GA model. The eciency of response of the model to a given set of antigens is studied, and an improvement is observed. The size of the "semi-elite" group a ects the eciency: it appears that an optimal size exists for a given set of antibodies and antigens. Based on this model, an extended model in which antibodies are developed by an individual who has evolved its gene segments in a given environment of antigens is proposed. Experimental results demonstrate that the interaction of evolution, gene recombination, and clonal selection is an e ective means of performing a multiple pattern recognition task when the given pattern has a certain structure behind it. One major issue which should be addressed in further research is a co-evolution of the virus and the immune system, in which antigens can be allowed to evolve so that they try to avoid to
6
6. Summary and Conclusion
4 antigen
Average number of antibodies per antigen
(b)
(a)
Time S teps
8 antigen
Average number of antibodies per antigen (a) (b) Time S teps
Figure 3: Comparison of eciency of response in the random environment between the (a)evolved and (b)unevolved individuals. 4 antigen
Average number of antibodies per antigen
(a) (b)
Time S teps
8 antigen
Average number of antibodies per antigen (a) (b)
Time S teps
Figure 4: Comparison of eciency of response in the structured environment between the (a)evolved and (b)unevolved individuals.
6. Summary and Conclusion
7
be matched by antibodies. This would not be unrealistic modeling as the mutation of antigens has been observed, sometimes in a very short time scale as in the case of HIV- 1[12,13]. Studies of such interactive model systems may lead us to other useful ideas in the construction of ecient information processing systems.
References [1] Theoretical Immunology, Parts I and II, Edited by A. S. Perelson, Addison-Wesley, Redwood City, CA, 1988 [2] A. S. Perelson, "Immune network theory", Immunology Review, 110, (1989), 5-36. [3] Theories of Immune Networks, H. Atlan and I.R. Cohen, Eds. (1989), Springer-Verlag, Berlin [4] J. Faro and S. Velasco, "Studies on a Recent Class of Network Models of the Immune System", J. Theor. Biol. 164, (1993) 271-290 [5] J. O. Kephart, "A biologically inspired immune system for computers" in Arti cial Life, edited by C. G. Langton, Addison-Wesley, Redwood City, CA, 1994 [6] S. Forrest, B. Jovornik, R. E. Smith and A. S. Perelson, "Using Genetic Algorithms to Explore Pattern Recognition in the Immune System", Evolutionary Computation, 1, 191-211, (1993) [7] R. Hightower, S. Forrest, and A. S. Perelson, "The evolution of secondary organization in immune gene libraries" in Proceedings of the Second European Conference on Arti cial Life, (1993) [8] J. D. Farmer, N. H. Packard, and A. S. Perelson, "The immune system, adaptation, and machine learning", Physica D 22 (1986) , 187-204 [9] J. D. Farmer, "A Rosetta Stone for Connectionism", Physica D 42, (1990)153-187 [10] H. Bersini and F. J. Varela, " The immune recruitment mechanism: a selective evolutionary strategy", in the Proceedings of ICGA-91, 520-526, (1991) [11] T. Ohira and H. Kitano, submitted. [12] X. Wei et al., "Viral dynamics in human immunode ciency virus type 1 infection", Nature, 373, (1995) 117-122 [13] D. D. Ho et al., "Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection, Nature, 373 (1995) 123-126
