Distributed Evolutionary Computation for Robotic

0 downloads 0 Views 3MB Size Report
Oct 31, 2018 - itself or even walk by moving the right pistons in a specific pattern. ...... Figure 8.9 - Graph of Bitwise Mutation Variance: Generations vs Fitness . ...... For example, the concept of a “cable”, could be added. .... [20] T. V. Mathew, “Genetic Algorithm,” Indian Institute of Technology ... string best_chromosome;.

School of Electrical Engineering, Computing and Mathematical Sciences

Distributed Evolutionary Computation for Robotic Design in Balancing and Movement

By Jered Geoffrey Masters

A thesis submitted for the degree of Bachelor of Engineering in Computer Systems Engineering November, 2018

ABSTRACT This thesis has the end goal of creating a software which is capable of designing hyper flexible and robust robots for NASA (and/or others) in space exploration and other terrain navigation problems. As part of their experiments into new frontiers NASA is investigating unconventional robots which exhibit a property called tensegrity. These tensegrity robots are made up of simple bars, cables and pistons and form a self-supporting structure with principals of tension and integrity. Examples include a kind of ball which can roll distorting itself or even walk by moving the right pistons in a specific pattern. NASA’s interest in these robots is that they are light-weight, low cost, robust, can survive large impacts with a payload, and can tackle terrain much rougher than conventional robotics. [1]

To achieve this goal program has been designed which uses evolutionary computation to design tensegrity robots. This program “builds” robots based different chromosomes where every robot has its own chromosome that completely defines its structure and material properties. The program then evaluates the robots and evolves the chromosomes to improve the performance in a continuous cycle. This is referred to as a Genetic Algorithm (GA)and is a well-documented and well used technique.

The specific design of the chromosome and robotics structure is intended to demonstrate emergence [2], where collections of simple rules and structures give rise to complex systems. The robotics used reduces all parts to either an edge or a vertex, referred to as a joint or muscle. These two building blocks have various parameters of their own which define how they behave and com together to build some very creative and complex robots.

i

This dissertation proposes some new ideas for improving GAs, investigates some existing ideas and compares against well-established ideas. The sub-goal is to make improvements in execution time and end-product performance.

The process of designing this program required tuning of many parameters of the GA (including the application of new and existing ideas mentioned above) for ideal performance. Testing and results of this tuning is presented here in this dissertation. The best (highest performing) parameters have been selected and then used in some final validation tests where the simulation contains obstacles. This it done to confirm the algorithm can design robots on demand which can handle obstacles.

Completing this task also required utilizing more computing resources and distributing the load. A small protocol was designed to allow different computers to all work on the problem, running simulations at the same time, and sharing the results to a central server for analysis. The GA and robotics simulation were built in C++ with OpenMP for multithreading.

This final output of this project is a software which can readily produce greatly varying designs for robots for different environments and obstacles. Many different variations of the GA were tested to find the optimum configuration for the given search space. The resulting algorithm was validated by testing it against several different environments and obstacles.

ii

Jered G. Masters 42 Todd Avenue Como, WA 6152 Wednesday, 31 October 2018

Prof Kit Yan Chan Senior Lecturer in Electrical Engineering, Computing and Mathematical Sciences. Curtin University Kent Street, Bentley GPO Box U1987 Perth WA 6845

Dear Prof Kit Yan Chan,

I, Jered Masters, hereby submit my thesis entitled “Distributed Evolutionary Computation for Robotic Design in Balancing and Movement.” as part of my requirements for completion of the Bachelor of Engineering in Computer Systems Engineering. I declare that this thesis is entirely my own work with the exception of the acknowledgements and references mentioned.

Yours sincerely,

Jered G. Masters 14308992

iii

DECLARATION OF PUBLISHED WORK I also acknowledge that parts of the progress report presented at the end of the first semester of this project have been used in the following chapters of this thesis, and the report has been properly referenced:



1.0 Introduction



3.0 GA Mechanism



4.0 GA Operations



5.0 Robotic Simulation



7.0 Implementation



8.0 GA Results

iv

ACKNOWLEDGEMENTS I would like to thank my supervisor Dr Kit Yan Chan from Curtin University for his guidance, expertise, and understanding. His insight, ideas and recommendations made this project possible. This project was enjoyable and fulfilling, and would not have been possible without his help.

I would like to thank Prof Iain Murray for his advice and assistance in course and time management. Getting this project over the line in a timely fashion would not have been possible without his input.

v

TABLE OF CONTENTS Abstract ................................................................................................................................... i Declaration Of Published Work ........................................................................................... iv Acknowledgements ............................................................................................................... v List of Figures ....................................................................................................................... xi List of Tables ...................................................................................................................... xiv List of Equations .................................................................................................................. xv 1.0 Introduction ..................................................................................................................... 1 1.1 Problem Statement ....................................................................................................... 1 1.2 Motivation of Problem ................................................................................................. 2 1.2.1 Motivation of Tensegrity Robotics ....................................................................... 2 1.2.2 Motivation of Genetic Algorithm.......................................................................... 2 1.3 Overview of Research .................................................................................................. 3 1.4 Summary of Results ..................................................................................................... 4 1.5 Chapter Summaries ...................................................................................................... 5 2.0 Literature Review ............................................................................................................ 7 2.1 Robotic Design ............................................................................................................ 7 2.1.1 Conventional Robotics .......................................................................................... 7 2.1.3 Complex Systems Emerging from Simple Rules .................................................. 7 2.1.4 Applications .......................................................................................................... 8 2.1.5 Materials ................................................................................................................ 9 2.1.6 Domain Randomisation ....................................................................................... 10 2.1.7 Incremental Feedback ......................................................................................... 10 2.2 Genetic Algorithms .................................................................................................... 11 2.2.1 Bitwise Mutation Variance ................................................................................. 13

vi

2.2.2 Steepest Descent.................................................................................................. 14 2.3 Slowly Increasing Complexity................................................................................... 15 3.0 GA Mechanism .............................................................................................................. 16 3.1 Overview .................................................................................................................... 16 3.2 Chromosome .............................................................................................................. 18 3.3 Crossover ................................................................................................................... 19 3.4 Selection..................................................................................................................... 20 3.5 Mutation ..................................................................................................................... 21 3.6 Fitness ........................................................................................................................ 22 3.7 Viability ..................................................................................................................... 23 3.7.1 Pre-simulation tests ............................................................................................. 23 3.7.2 Generation Zero .................................................................................................. 23 4.0 GA Operations ............................................................................................................... 24 4.1 Steepest Descent ........................................................................................................ 24 4.2 Selection Pressure ...................................................................................................... 26 4.3 Crossover Slicing ....................................................................................................... 27 4.4 Mutation Rate ............................................................................................................ 28 4.5 Linear Mutation Variance .......................................................................................... 30 4.6 Bitwise Mutation Variance ........................................................................................ 31 5.0 Robotic Simulation ........................................................................................................ 32 5.1 Robots ........................................................................................................................ 32 5.2 Objective / Success Measure ..................................................................................... 34 5.3 Search Space .............................................................................................................. 35 6.0 Robotic Optimisation..................................................................................................... 36 6.1 Duration Variance ...................................................................................................... 36

vii

6.2 Fitness Calculation ..................................................................................................... 38 6.2.1 Distance Travelled .............................................................................................. 38 6.2.2 Height and Distance Travelled ............................................................................ 39 6.2.3 Time Average Height and Distance .................................................................... 40 6.3 Randomised Physics .................................................................................................. 41 7.0 Implementation .............................................................................................................. 42 7.1 Technology ................................................................................................................ 42 7.2 Parallel Processing ..................................................................................................... 43 7.3 Distributed Computing .............................................................................................. 44 7.3.1 Dispatching ......................................................................................................... 45 7.3.2 Timing and Benchmarking .................................................................................. 46 8.0 GA Results..................................................................................................................... 47 8.1 Fitness Distribution .................................................................................................... 47 8.2 Effects of Population Size.......................................................................................... 49 8.3 Effects of Duration Variance ..................................................................................... 50 8.4 Mutation Variance ..................................................................................................... 52 8.4.1 No Mutation Variance ......................................................................................... 52 8.4.2 Linear Mutation Variance ................................................................................... 54 8.4.3 Bitwise Mutation Variance ................................................................................. 56 8.5 Selection Pressure ...................................................................................................... 58 8.6 Crossover Slicing ....................................................................................................... 60 9.0 Robotic Results .............................................................................................................. 62 9.1 Fitness Calculation ..................................................................................................... 62 9.1.1 Distance Travelled .............................................................................................. 63 9.1.2 Height by Distance Travelled .............................................................................. 65

viii

9.1.3 Time Averaged Height by Distance Travelled.................................................... 67 9.2 Ramp Obstacle ........................................................................................................... 69 10.0 Discussion.................................................................................................................... 70 10.1 Crossover Slicing ..................................................................................................... 70 10.2 Selection Pressure .................................................................................................... 70 10.3 Mutation Variance ................................................................................................... 71 10.4 Duration Variance .................................................................................................... 71 10.5 Steepest Descent ...................................................................................................... 72 10.6 Domain Randomisation ........................................................................................... 72 10.7 Cross-Cutting Parameters ........................................................................................ 72 10.8 Developmental Stages.............................................................................................. 73 11.0 Conclusion ................................................................................................................... 74 12.0 Future Work................................................................................................................. 76 12.1 Enhance Robotic Simulation ................................................................................... 76 12.2 Effect of Domain Randomisation ............................................................................ 76 12.2 Neural Network ........................................................................................................ 77 13.0 References ................................................................................................................... 79 Appendix A – Updated Plan ................................................................................................ 83 Appendix B – Simulation.cpp ............................................................................................. 84 Appendix C – Ga.cpp .......................................................................................................... 88 Appendix D – Robot.cpp ..................................................................................................... 93 Appendix E – Muscle.cpp ................................................................................................... 99 Appendix F – Joint.cpp...................................................................................................... 100 Appendix G – Chromosome.cpp ....................................................................................... 101 Appendix H – Parameters.cpp ........................................................................................... 103

ix

Appendix I – Communicator.cpp ...................................................................................... 104 Appendix J – Renderer.cpp ............................................................................................... 107 Appendix K – ApiController.php ...................................................................................... 110 Appendix L – TurnItIn Report........................................................................................... 113

x

LIST OF FIGURES Figure 1.1 - Summary of Robotic Results ............................................................................. 4 Figure 2.1 - NASA Planetary Lander. Source: [4] ................................................................ 8 Figure 2.2 - Metal Foam demonstrating shape memory under heat. Source: [11] ................ 9 Figure 2.3 - Genetic Algorithm Process, Source: [6] .......................................................... 12 Figure 2.4 - Genetic Crossover, Source: [6] ........................................................................ 12 Figure 2.5 - Bit mutation rate over both iterations and bit significance, Source: [15] ........ 13 Figure 2.6 - Hybrid Genetic Algorithm and Gradient Descent. Source: [16] ..................... 14 Figure 2.7 - Slowly Increasing Complexity. Source: [18] ................................................... 15 Figure 3.1 - GA Flow Chart................................................................................................. 16 Figure 3.2 - Genetic Algorithm UML. Source: [19]............................................................ 17 Figure 3.3 - Genetic Segment Composition ........................................................................ 18 Figure 3.4 - Genetic Crossover ............................................................................................ 19 Figure 3.5 - Selection Roulette Wheel................................................................................. 20 Figure 4.1 - Steepest Descent .............................................................................................. 24 Figure 4.2 – Example of Steepest Descent in Non-convex Search Space ........................... 25 Figure 4.3 – Selection Roulette Wheel for Selection Pressures 1 & 2 & 3 ......................... 26 Figure 4.4 - Crossover Slicing Examples 1 & 4 .................................................................. 27 Figure 4.5 - Mutations vs probability for default probability .............................................. 28 Figure 4.6 - Mutation Rate: Mutations vs Probability ......................................................... 29 Figure 4.7 - Linear Mutation Variance ................................................................................ 30 Figure 4.8 - Bitwise Mutation Variance .............................................................................. 31 Figure 5.1 – Robot force diagram ........................................................................................ 32 Figure 5.2 - Robot pushing itself uphill with a rear node .................................................... 33 Figure 5.3 – Example of a Non-Convex Search Space ....................................................... 35

xi

Figure 6.1 - Duration Variance: Run Time vs Generation Ratio......................................... 36 Figure 7.1 - Multithread Loop ............................................................................................. 43 Figure 7.2 - Distributed Computing Model ......................................................................... 44 Figure 8.1 - Fitness Distribution without Steepest Descent ................................................ 47 Figure 8.2 - Fitness Distribution with Steepest Descent ..................................................... 48 Figure 8.3 - Graph of Fitness vs Generations for various Population Sizes........................ 49 Figure 8.4 - Graph of Fitness vs Generations for Duration Variance.................................. 50 Figure 8.5 - Graph of Runtime vs Generations for Duration Variance ............................... 50 Figure 8.6 - Graph of Duration Variance: Generations vs Scaled Fitness .......................... 51 Figure 8.7 - Graph of Mutation Variance: Generations vs Fitness ...................................... 52 Figure 8.8 - Graph of Linear Mutation Variance: Generation vs Fitness ............................ 54 Figure 8.9 - Graph of Bitwise Mutation Variance: Generations vs Fitness ........................ 56 Figure 8.10 - Graph of Select Pressure: Generations vs Fitness ......................................... 58 Figure 8.11 - Graph of Crossover Slicing: Generations vs Fitness ..................................... 60 Figure 9.1 - Distance Travelled, Generation 1 .................................................................... 63 Figure 9.2 - Distance Travelled, Generation 50 .................................................................. 63 Figure 9.3 - Distance Travelled, Generation 100 ................................................................ 64 Figure 9.4 - Distance Travelled, Generation 150 ................................................................ 64 Figure 9.5 - Distance Travelled, Generation 200 ................................................................ 64 Figure 9.6 - Height by Distance Travelled, Generation 1 ................................................... 65 Figure 9.7 - Height by Distance Travelled, Generation 50 ................................................. 65 Figure 9.8 - Height by Distance Travelled, Generation 100 ............................................... 66 Figure 9.9 - Height by Distance Travelled, Generation 150 ............................................... 66 Figure 9.10 - Height by Distance Travelled, Generation 200 ............................................. 66 Figure 9.11 - Time Averaged Height by Distance Travelled, Generation 1 ....................... 67

xii

Figure 9.12 - Time Averaged Height by Distance Travelled, Generation 50 ..................... 67 Figure 9.13 - Time Averaged Height by Distance Travelled, Generation 100 ................... 68 Figure 9.14 - Time Averaged Height by Distance Travelled, Generation 150 ................... 68 Figure 9.15 - Time Averaged Height by Distance Travelled, Generation 200 ................... 68 Figure 9.16 - Population of Robots attempting ramp obstacle ............................................ 69 Figure 12.1 - Robotic Enhancement Neural Network ......................................................... 77

xiii

LIST OF TABLES Table 1.1 - Summary of GA Results ..................................................................................... 4 Table 3.1 - Mutation Bitmask Example .............................................................................. 21 Table 6.1 - Domain Randomisation Scaling Factors ........................................................... 41 Table 7.1 - Programming Languages Energy, Time and Memory usage, Source: [26] ...... 42 Table 8.1 - Population vs Generations Results .................................................................... 49 Table 8.2 - Duration Variance vs Scaled Fitness Results .................................................... 51 Table 8.3 - No Mutation Variance: Mutation Rate vs Fitness Results ................................ 53 Table 8.4 - Linear Mutation Variance: Mutation Rate vs Fitness Results .......................... 55 Table 8.5 - Bitwise Mutation Variance: Mutation Rate vs Fitness Results ......................... 57 Table 8.6 - Selection Pressure vs Fitness Results ................................................................ 59 Table 8.7 - Crossover Slicing vs Fitness Results ................................................................ 61

xiv

LIST OF EQUATIONS Equation 3.1 - Fitness Calculation....................................................................................... 22 Equation 4.1 - Selection Pressure: Scaled Fitness ............................................................... 26 Equation 4.2 - Default probability of mutation of a single bit ............................................ 28 Equation 4.3 – Scaled probability of mutation of a single bit ............................................. 29 Equation 4.4 - Linear Mutation Calculation ........................................................................ 30 Equation 4.5 - Bitwise Mutation Calculation [3] ................................................................ 31 Equation 5.1 - Fitness Calculation....................................................................................... 34 Equation 5.2 - Problem Search Space ................................................................................. 35 Equation 6.1 - Distance Travelled Fitness ........................................................................... 38 Equation 6.2 - Final Height and Distance Fitness ............................................................... 39 Equation 6.3 - Fitness Calculation....................................................................................... 40 Equation 9.1 - Distance Travelled Fitness ........................................................................... 63 Equation 9.2 - Final Height and Distance Fitness ............................................................... 65 Equation 9.3 - Fitness Calculation....................................................................................... 67

xv

1.0 INTRODUCTION The goal of this project is to design a technique and a software which can reliably produce designs for unconventional robots for various environments and obstacles. These robots should be relatively simple and capable of balancing and locomotion [3]. This task will explore various optimisation and tuning mechanisms for genetic algorithms (GAs).

The process is to provide very simple parts and components for the GA to build with, and encourage a complex, interconnected design to emerge. As mentioned in the next chapter, there are many examples from nature where staggering complexity emerges from simplicity. This presented its own challenge; designing a chromosome which can readily mutate without rapid degradation, allow a range of structures and expressions without being hopelessly unstable.

1.1 Problem Statement With unconventional robotics, the possible number of designs and configurations is almost limitless, and each scenario to which they could be applied would be ideally suited to a different design. Creating a new design for each environment by hand would require a lot of time and money, and would be a very narrow, incomplete search of the solution space.

Design a system which uses GAs to design a prototype robot which utilises the principals of tensegrity to balance, move around a given environment and overcome given obstacles.

1

1.2 Motivation of Problem NASA has taken special interest in these tensegrity robots and is exploring designs and configurations for space exploration [4]. These robots are likely going to play some role in future exploration missions and it is therefore of interest to be able to rapidly design them using effective learning techniques.

1.2.1 Motivation of Tensegrity Robotics Tensegrity robotics allows robots to quickly adapt to new or changing environments and they’re inherently extremely tolerant of physical impacts and damage. This simpler design also allows novel features to emerge without complex design techniques. [5] It is also much cheaper to manufacture tensegrity robotics as their inherent simplicity makes assembly much easier. [4]

1.2.2 Motivation of Genetic Algorithm This problem has a non-convex search space (explained in 5.3 Search Space, page 35). GAs are very good at navigating this type of problem,

as explained in 2.2 Genetic Algorithms (page 11). This is because they exhibit a continuous modify-evaluate cycle and introduce an important element of randomness, allowing a large degree of variation, covering a lot of solutions from the search space at once.

The problem is also simple to describe and model and has parameters which give scope to the mutation and crossover process. The design process of the robots also gives generates interesting modelling questions (about structure and function). These qualities make this problem a good target for GAs [6].

2

1.3 Overview of Research There has been significant research done on GAs, modifying them in various ways and introducing new novel ideas. This thesis leans on some of this research to adapt these ideas, as well as machine learning research in general. This thesis also draws a comparison between conventional GAs and these modified concepts.

There has also been significant research into robotic design and unconventional robotics such as tensegrity robotics. This research is paving the way to a new generation of robots and possibilities for exploration and production, and informs some of the techniques in this thesis.

3

1.4 Summary of Results The best configuration for the genetic algorithm is as follows: Parameter

Value

Impact

Population

200

High

Generations

200

Medium

Mutation Variance

Linear Variance

Medium

Mutation Rate

5

High

Crossover Slicing

2

Low

Steepest Descent

Off

Medium

Duration Variance

Off

Medium

Table 1.1 - Summary of GA Results

The best fitness evaluation for the robots is the Time Averaged Height by Distance Travelled. This is a fitness evaluation which takes into account all the nodes (joints) in the robot, averages a fitness value across the entire duration of the simulation, instead of simply looking at the final state of the robot.

Figure 1.1 - Summary of Robotic Results

These two results enable the final software product which will design robots based on a given terrain and obstacles.

4

1.5 Chapter Summaries 1.0 Introduction

General summary of work and report.

2.0 Literature Review

Review of existing literature relevant to this report. Research topics include robotics, tensegrity robotics, GAs, complexity theory, smart materials.

3.0 GA Mechanism

Explanation of the fundamental processes in GA and how they are applied in this experiment.

4.0 GA Operations

New concepts and processes applied to the GA in this experiment. This refers to the parameters of the GA and the way these parameters affect the algorithm.

5.0 Robotic Simulation

Overview of the robotic simulation used and explanation of details relevant to the selection process.

6.0 Robotic Optimisation

Several optimisations are possible for improving the robots and simulation to produce better performing robots. This chapter looks at different fitness calculation methods and randomising the physics of the simulation.

5

7.0 Implementation

How the experiment was constructed and executed. This explains the technologies used and interesting techniques employed for greater efficiency.

8.0 GA Results

Results and analysis of the experiment. Many different tests were performed with different testing parameters. This chapter contains the summarised results and analysis of each set of parameters.

9.0 Robotic Results

Investigation into which fitness calculation is best and using the learnings from the experiment to tackle problems more similar to the real world. The algorithm is tested against various environments and obstacles.

10.0 Discussion

Discussion of results and the experiment, and the meaning/implications of the results.

11.0 Conclusion

Conclusion of the report. Covering objectives, outcomes and learnings.

12.0 Future Work

Novel ideas for study based on this work.

6

2.0 LITERATURE REVIEW 2.1 Robotic Design 2.1.1 Conventional Robotics Conventional robotics, or hard robotics, relies on strong and precise control of all parts of the robot and preventing almost all else from moving. This very rigid view of the system presents many inherent issues which must be intentionally designed against, largely complicating the design, construction, and tuning. These problems include low tolerance for an unstable environment, low tolerance for a dynamic environment, low tolerance for imprecise movement, low tolerance of manufacturing fault or damage. [7]

2.1.3 Complex Systems Emerging from Simple Rules Emergence is the name given to the phenomenon where a small set of simple rules gives rise to a large and complex system. [8] These systems can achieve difficult tasks thought only possible by designing very complex and interconnected rules from the top most level down. In his editorial, Chemistry Nobel Laureate Fraser Stoddart commented that it is advantageous to embrace the intellectual challenge of the complex nature of emergence and emergent systems. [2]

7

2.1.4 Applications A report out of Cornell University for NASA concluded soft robots have many advantages over traditional robotics in underwater conditions. These advantages include the ability to expand or collapse it body for efficient transport and given the materials used, better inherent radiation resistance and more tolerance for extreme temperatures. [9]

A report out of the NASA Ames Research Center [4] show cases a planetary lander which has research funding for exploration missions. Figure 1 below shows how a proposed space module could drop several onto a planet surface out much concern for physical damage.

Figure 2.1 - NASA Planetary Lander. Source: [4]

8

2.1.5 Materials A multi-objective hydraulic cylinder has been researched and designed for the purpose of implementation in soft robotics by Nestor F. Michelena and Alice M. Agogino. This hydraulic cylinder optimises three properties which are important for soft robotics; crosssectional area, circumferential stress ratio and pressure ratio. [10]

Metal Foam Hybrid [11] is a material being designed by Rob Shepherd at Cornell University, partially funded by the U.S. Airforce Office of Scientific Research, for the purpose of, among other thing, soft robotics. This metallic material can have varying rigidity depending on its temperature, and when sufficiently heated, it will return its original shape.

Figure 2.2 - Metal Foam demonstrating shape memory under heat. Source: [11]

9

2.1.6 Domain Randomisation In their paper on Learning Dexterous In-Hand Movement, Marcin Andrychowicz et al explore the use of domain randomisation in machine learning. [12] While attempting to train a robotic hand to manipulate a wooden block, a simulation is created. However, it is a common problem in machine learning where a simulation doesn’t perfectly match reality, so whatever is learnt by the algorithm is of little use. The team overcomes this problem by training the algorithm on thousands of different simulations all with slightly different physics. By varying the effect of gravity, friction and inertia of the block, friction of the hand, the team trains the algorithm to correctly manipulate the block in a way that generalises to the real world much better.

2.1.7 Incremental Feedback Another common issue with machine learning is providing the learning agent with feedback about it’s performance. While attempting to train a neural network to play computer games, Stadie et al had significant issues training for a game where the score updates very infrequently. [13] The team was training the algorithm using the raw game score as feedback for the different games. A particular game only updated the score at the end of a long series of complicated movements (puzzle solving), the AI rarely accomplished the task and when it did it was very difficult to work what it did well. The solution to issue was to create other feedback for the AI to guide it towards the goal.

10

2.2 Genetic Algorithms Genetic Algorithms (GAs) were introduced in 1975 by John Holland in his book “Adaptation in Natural and Artificial Systems” [14], as a mechanism for searching an extremely large, high-dimensional search space by mimicking natural evolutionary processes. In their paper “Developing a Generic Genetic Algorithm” [6], Neville and Sibley discuss the ability of species to evolve towards a complex and high functioning systems without the need of a supervisor, and as such this progress is automatic. Neville and Sibley lay out some steps for developing a GA.

The core tenants of a GA are mutation, crossover, evaluation and selection:

Mutation

Crossover

The process of randomly altering tiny parts of a chromosome.

The process of mixing (similar to breeding) two chromosomes together to create a new chromosome (child).

Evaluation

The process of setting the chromosome against the task being solved and giving it a score to measure how successful it is.

Selection

The process of ranking a selecting the chromosomes to crossover to create the next generation.

These processes are shown in a high-level overview of GAs below

11

Figure 2.3 - Genetic Algorithm Process, Source: [6]

Figure 2.4 - Genetic Crossover, Source: [6]

12

2.2.1 Bitwise Mutation Variance Varying the rate of mutation in GAs can have a positive impact on the final fitness of an experiment, as shown by Kit Yan Chan, M. Emin Aydin and Terence C. Fogarty [15]. Their study looked at ranking the bits in terms of their impact on the final fitness and varying the mutation rate of those bit over the course of 300 generations. In this paper, this is referred to bitwise mutation variance.

Figure 2.5 - Bit mutation rate over both iterations and bit significance, Source: [15]

13

2.2.2 Steepest Descent Combining GA and gradient descent can create an algorithm which performs better than either GA or GD by themselves. This was shown by Maria E. Requena-Pérez, Antonio Albero-Ortiz, Juan Monzó-Cabrera, and Alejandro Díaz-Morcillo [16]. Their paper looked at running a GA until it locates the zone of the global maximum, then starting a new GA where the entire population is copied from the fittest chromosome of the previous GA. This process is described by Figure 2.6 below.

Figure 2.6 - Hybrid Genetic Algorithm and Gradient Descent. Source: [16]

14

When considering higher dimensional search problems, the frequency of local minima and maxima decrease, and the frequency of saddle points increases. This is somewhat intuitive; when there are more dimensions, the probability at any point there are no dimensions with a gradient is low. This was proven by Carl E. Rasmussen in 2006 [17]. Considering this, it is worthwhile to favour a technique that uses stochastic search and have little concern for local maxima problems.

2.3 Slowly Increasing Complexity As Tero Karras et al showed in their paper Progressive Growing Of Gans For Improved Quality, Stability, And Variation [18], slowly increasing the complexity of a problem while it is being learnt and solved by a machine can have positive results. The team started with a neural network producing a 4x4 pixel image, and slowly increased the required resolution by adding layers to the neural network. This resulted in very high quality, high resolution generated images. This kind of network would have taken an impractically long time to train with classic techniques.

Figure 2.7 - Slowly Increasing Complexity. Source: [18]

15

3.0 GA MECHANISM 3.1 Overview The GA implemented has a process which closely follow the industry standard for GAs. [6] This is detailed below on the flow chart and UML below. To see any of the parts of this chapter implemented in code see Appendix C – Ga.cpp (page 88).

Figure 3.1 - GA Flow Chart

16

Figure 3.2 - Genetic Algorithm UML. Source: [19]

17

3.2 Chromosome The chromosome of the robot is split up into 16 bit segments; each segment defines a single building block of the robot. These segments are also divided into 4 bit chunks or nibbles. The type of component is defined by the value of the first nibble, the allocation of which can be seen in the second row of the table below.

Figure 3.3 - Genetic Segment Composition

In the table above, where an index is mentioned, it has a modulo applied of the size of the given vector it references. For example; a muscle must connect to two nodes, if it gets the values 2 and 11 but there is only 8 nodes, it will join to nodes 2 and 3.

18

3.3 Crossover Crossover is the means for combining chromosomes for the next generation from the previous generation. For every two chromosomes in the next generation, two chromosomes are selected from the previous (excluding the case of zero crossover). These two older chromosomes are “mixed” together, by selecting random sections of each and stitching them together. This selection process is done so that any chunk is in the same position as it was in the parent chromosome.

Figure 3.4 - Genetic Crossover

The black vertical lines in Figure 3.4 above represent crossover slice, locations where the chromosome gets sliced, where the new chromosome stops reading from one parent, and starts reading from the other. The location of these crossover points is random; however, the frequency is varied in different experiments. For more information see 4.3 Crossover Slicing, page 27.

Crossover is intended to be part of the trade-off between convergence and avoiding local maxima/minima. When two chromosomes crossover, there are effectively two convergence points placed between them in the search space where the new chromosomes are. This process will slowly cause the population to converge on a single solution (or within a maximum delta of a single solution).

19

3.4 Selection All chromosomes in the previous generation are evaluated for fitness and this fitness value determines how likely a chromosome is to be selected for crossover. The chromosomes are placed on a roulette wheel as shown below in Figure 4.3, and the wheel is “spun” with the better performing chromosomes more likely to be selected. [20]

Figure 3.5 - Selection Roulette Wheel

In Figure 3.5 above, chromosome 6 is most likely to be chosen out of the 10 chromosomes.

This roulette wheel technique was selected because it is an industry standard. [6] [20]

20

3.5 Mutation After a new generation of chromosomes have been selected and crossed-over, they are mutated. This mutation occurs by flipping random bits in in the chromosome. For each 16 bit segment of genetic information, a 16 bit bitmask is generated and an XOR (exclusive or) is performed which will flip each bit where there is a corresponding 1 in the bit mask.

Genetic Segment

1011010100111000

Bitmask

0001000000010010

Result

1010010100101010

Table 3.1 - Mutation Bitmask Example

21

3.6 Fitness The fitness of each robot is calculated by averaging the product of each node x y values every millisecond, then averaging that value over the number of milliseconds. [3].

where

m : total milliseconds of simulation n : total number of nodes in robot node[i][t].x : x position of node i at time t node[i][t].y : y position of node i at time t Equation 3.1 - Fitness Calculation

This is selected to maximise the incentives for traversing to the right and maintaining stability. This favours a shorter walker which has walked further over a tall one which hasn’t moved very much. It also favours a taller walker which has travelled the same distance as a shorter one [3]. This is calculated by summing the product of each nodes x and y position, described in Equation above.

By constructing the fitness in this way many hidden pitfalls are avoided. Robots which are lying on or close to the ground score very low fitness and robots which are very tall but don’t move very far also score very low fitness. The incentive is to get as many nodes as high as possible and as far to the right as possible.

These pitfalls and other fitness calculations are discussed further in 6.2 Fitness Calculation, page 38.

22

3.7 Viability 3.7.1 Pre-simulation tests When a robot is constructed from the chromosome, it is put through a series of checks and tests before simulation begins to check if it’s a viable subject. Failing the following tests will “kill” (remove from simulation and score 0 in fitness):



The chromosome yields zero nodes



The chromosome yields zero muscles



The chromosome yields more nodes than muscles



Any muscle attaches to the same node at both ends

Failing these tests means the offspring is “illegal” and is penalised. [6]

3.7.2 Generation Zero In the first generation of robots, it is guaranteed that all robots have passed pre-simulation tests. This improves the performance by not starting the simulation and GA with an incomplete population.

To guarantee a viable population, the chromosomes are generated, assembled and discarded if they are not viable.

Every subsequent generation will simply begin simulation with an incomplete population in the case of an inviable chromosome. This presents the issue of an empty population.

23

4.0 GA OPERATIONS 4.1 Steepest Descent Mechanism for aggressively pursuing the steepest descent at any point in the search. Quite simply it means the most successful chromosome in a population gets copied into the next generation without any mutation [3] [21].

Figure 4.1 - Steepest Descent

This mechanism prevents regression caused by random mutations. As shown in Figure 4.2 below, the search cannot move uphill, away from the goal (the minima). This is, in effect, applying simulated annealing to the algorithm, or monotonicity. This can be especially advantageous where high mutation rates are frequently causing destructive changes which wipe out a population, see 6.2.2.1 Empty Generation (page 39).

24

Figure 4.2 – Example of Steepest Descent in Non-convex Search Space

25

4.2 Selection Pressure Selection Pressure is a mechanism for giving the better performing chromosomes an increased edge. For example, when Selection Pressure is set to 2, all the fitness values are squared before the roulette wheel is constructed, as shown in Figure 4.3. If selection pressure is set to 3, the fitness values are cubed. As shown in Equation below, using the selection pressure as an exponent in this way, causes the better performing chromosomes to have a larger fitness relative to the total fitness of all the chromosomes.

Equation 4.1 - Selection Pressure: Scaled Fitness

Figure 4.3 – Selection Roulette Wheel for Selection Pressures 1 & 2 & 3

The figure above shown a constructed roulette wheel for the same 10 chromosomes, with a selection pressure of 1, 2 and 3. It can be observed that chromosome 6, the best performer, goes from 20% chance of selection, to almost double at 38%.

26

4.3 Crossover Slicing Crossover slicing defines the number of slices performed during the crossover process. With higher crossover slicing, the chromosome is more sliced and fragmented. [22]

Figure 4.4 - Crossover Slicing Examples 1 & 4

This quantity of slicing controls how close to the center of convergence the new chromosomes are, but also controls the possible negative impact of crossing over. By increasing the slicing number, the new chromosome will be a more blended mix of the two parent chromosomes, but it’s also more likely that some critical structure in the chromosome has been altered or destroyed.

27

4.4 Mutation Rate This is a variable which changes the rate of mutation after crossover. For each bit in the chromosome the chance of mutation is proportional to the mutation rate.

Equation 4.2 - Default probability of mutation of a single bit

This number (320) was initially set to 16 to match the 16 bit unit for components. However early testing showed this to be extremely unstable and the value was then tuned to improve the results and expose some interesting differences in the solution space. Using the value of 320 brings the likely minimum mutation rate for a whole chromosome to one.

Figure 4.5 - Mutations vs probability for default probability

28

This mutation probability is then varied using the mutation rate and mutation variance. [6]

where

mr : mutation rate (0 – 1) mv : mutation variance scale factor (0 – 2)

Equation 4.3 – Scaled probability of mutation of a single bit

As mutation rate is varied the likely amount of mutations in the whole chromosome changes. This is shown below in Figure 4.6. The probability of mutation is also changed by the mutation variance in the variable mv. This is explained in the next two sections.

Figure 4.6 - Mutation Rate: Mutations vs Probability

29

4.5 Linear Mutation Variance Have the mutation rate decrease linearly during the simulation. This happens by multiplying the specified mutation rate by floating point number from 2 to 0. This doubles the desired the mutation rate at the start of the simulation and reduces it to zero in the last generation. [3]

Figure 4.7 - Linear Mutation Variance

As shown in the Figure 4.7 above, and Equation 4.4 below, the mutation rate change decreases over the course of the generations and is not affected by the bit significance. The result of this is the chromosome gets more and more stable the closer the experiment gets to completion. This will theoretically bring out fitter chromosomes as larger mutations near maxima can bring radical regressions.

where

mv : mutation variance scale factor (0 – 2) rg : generation completion ratio (0 – 1) Equation 4.4 - Linear Mutation Calculation

30

4.6 Bitwise Mutation Variance Varying the mutation rate by both generation and bit significance. At the start of the simulation the most significant bits mutate more frequently, and the least significant bits mutate less frequently. By the end of the simulation the MSB mutate very rarely and the LSB much more often [3] [15].

Figure 4.8 - Bitwise Mutation Variance

where

mv : mutation variance scale factor (0 – 2) rg : generation completion ratio (0 – 1) b : bit significance (MSB – LSB) (0 – 3) Equation 4.5 - Bitwise Mutation Calculation [3]

31

5.0 ROBOTIC SIMULATION To evaluate the fitness of the chromosomes developed and validate the results of this paper, a robotic simulation was developed. This simulation would test a whole generation of robots at once and render a visual representation of the output to the screen.

5.1 Robots The robots have soft elastic sections which vary in length and rigidity, referred to as muscles. The muscles are joined at nodes which vary in mass, each node having at least two muscles. These two building blocks form the structure of the robots. [23]

The strength and desired length of the muscles, and the weight of the nodes vary through mutation and remain constant during the simulation.A small proportion of the muscles, approximately 20%, will mutate an oscillation function which varies the desired length of the muscle.

Figure 5.1 – Robot force diagram

32

Figure above, shows the structure and various forces applied to a robot. Weighted nodes are shown as coloured dots at the intersections of lines, muscles are shown as coloured lines joining nodes. Nodes which are a lighter green are lighter in mass, and nodes which are darker are heavier. Muscles which are a lighter red are weaker, and muscles which are a darker are stronger.

This specific robot exhibits a locomotion behaviour by oscillating node 3 around node 2 in a circular motion, causing the whole structure to bounce and move forward.

Figure 5.2 - Robot pushing itself uphill with a rear node

The robot in the Figure 5.2 above also exhibits locomotion by oscillating node 3 around node 2 in a circular motion. This time, it moves forward by pushing off the ground behind it. This was so successful; the robot was able to “walk” uphill. 33

5.2 Objective / Success Measure The robotic simulation of the GA chromosome demands robot be built which can maintain its own stability and “walk” a distance to the right on the screen in the allocated 15 seconds [3]. This incentivises speed and stability. This objective is also augmented later by placing obstacles in the path of the walkers.

where

m : total milliseconds of simulation n : total number of nodes in robot node[i][t].x : x position of node i at time t node[i][t].y : y position of node i at time t Equation 5.1 - Fitness Calculation

To quantify the success of the robot and encourage the goals described above, a fitness value is calculated by summing the product of each nodes x and y position. This is shown in the equation above.

34

5.3 Search Space The search space of this problem has 80 dimensions and 16 possibilities for each dimension:

1680 = 2.13 × 1096 Equation 5.2 - Problem Search Space

The problem also has a non-convex search space. [24] This means the downward slope of most of the points in the space does not lead to a global minimum (or maximum). Figure 5.3 below shows a 2-dimensional, non-convex search space for illustrative purposes only.

Figure 5.3 – Example of a Non-Convex Search Space

In a non-convex search space, there is no obvious path or algorithm which will reliably navigate from any point on the surface to the global minima (or maxima).

35

6.0 ROBOTIC OPTIMISATION 6.1 Duration Variance This is an optimisation technique to speed up the simulations. The earlier generations are expected to produce a chromosome which does not move very much, if at all. This means there’s no need to run the simulation to it’s full (time) completion for those generations. Using duration variance, the simulation runs for a much shorter time during the earlier generations, then duration increases linearly for the later generations [3].

Figure 6.1 - Duration Variance: Run Time vs Generation Ratio

Duration Variance also optimises the different developmental stages which are observed in the evolution of the robots. [25] For more discussion on the observed developmental stages phenomenon, see 10.8 Developmental Stages, page 73. This encourages a more stable development pattern and also allows more variation as the early stages are more evenly scored. [18]

36

This concept was employed to mimic the incremental feedback concept (see 2.1.7 Incremental Feedback, page10). [13] When not averaging the fitness value over time (see 6.2.2 Height and Distance Travelled, page 39), If all the robots fall over by the end of the simulation, the GA will treat them all as having the same fitness of zero. Therefore any robots which stayed up longer than the others would not be rewarded for that. This conceptually scales the required time to “survive” from very short up to the normal length. Any robots which are successful only for a short time are selected for and as the simulation lengthens (over the generations) “survival” becomes more demanding.

37

6.2 Fitness Calculation 6.2.1 Distance Travelled This is the simplest fitness calculation, found by using the average x position of the nodes in a robot at the end of the simulation. The average value is taken because the robots do not have a fixed number of nodes and the GA shouldn’t favour robots that simply have more nodes.

where

n : total number of nodes in robot node[i].x : x position of node i Equation 6.1 - Distance Travelled Fitness

This fitness function was initially used to minimise the impact of unintended side effects and allow the GA to find the most effective way to move across the screen without guiding towards a specific goal. This did have unintended consequences where the GA had no preference for robots which maintained their upright position. The GA even favoured robots which fell over in the desired direction at the end of the simulation as this increased their x values.

38

6.2.2 Height and Distance Travelled This is calculated by averaging the product of the x and y position of each node. This means that a node on the floor is worth zero (because it’s y value is zero), no matter how far it travels. A node which is high in the air however is worth a much more and a node only at a small height. A robot which falls over scores zero fitness and doesn’t have any chance of being selected for the next generation.

where

n : total number of nodes in robot node[i].x : x position of node i node[i].y : y position of node i Equation 6.2 - Final Height and Distance Fitness

6.2.2.1 Empty Generation This fitness function presents a problem however. If all the robots fall over and score zero, then there are no robots to choose from in the roulette wheel. This is referred to in this thesis as an empty generation. This can be solved by selecting from all the robots with equal likelihood, or starting with a fresh batch of random chromosomes. Neither of these solutions yielded good results and small populations or high mutation rates caused the entire test to continuously restart, preventing the classic influence of GAs on slowly improving a population.

39

6.2.3 Time Average Height and Distance The fitness of each robot is calculated by averaging the product of each node x y values every millisecond, then averaging that value over the number of milliseconds. [3].

where

m : total milliseconds of simulation n : total number of nodes in robot node[i][t].x : x position of node i at time t node[i][t].y : y position of node i at time t Equation 6.3 - Fitness Calculation

This is selected to maximise the incentives for traversing to the right and maintaining stability. This favours a shorter walker which has walked further over a tall one which hasn’t moved very much. It also favours a taller walker which has travelled the same distance as a shorter one [3]. This is calculated by summing the product of each nodes x and y position, described in Equation below.

By constructing the fitness in this way many hidden pitfalls are avoided. Robots which are lying on or close to the ground score very low fitness and robots which are very tall but don’t move very far also score very low fitness. The incentive is to get as many nodes as high as possible and as far to the right as possible.

This is the fitness function used for all the tests in the results section unless stated otherwise.

To see this final fitness function implemented, see Appendix D – Robot.cpp (page 93)

40

6.3 Randomised Physics To create robots which are more likely to perform in the real world, the robots were tested against many different versions of the simulation. This is an implementation of domain randomisation. By varying the physics randomly between simulations and testing each robot on many versions, then averaging it’s fitness across the simulation a more performant robot is created.

Domain randomisation is needed because no simulation perfectly matches reality and by testing the robots in many different versions of the simulation, a more generalised robot (therefore generalised solution) is created. [12] This is based on work done by Marcin Andrychowicz et al, see 2.1.6 Domain Randomisation (page 10).

The physical parameters which were randomised are listed below. In the testing each robot was subjected to 100 different simulations, with each robot getting it’s own randomised set every generation.

Parameter

Scaling Range

Gravity

0.1 – 2

Air Friction

0.1 – 10

Ground Friction

0.1 – 2

Ground Hardness

0.1 – 10

Muscle Elasticity

0.1 – 2

Inertia

0.1 – 2

Table 6.1 - Domain Randomisation Scaling Factors

To see this implemented see Appendix D – Robot.cpp (page 93).

41

7.0 IMPLEMENTATION 7.1 Technology The primary program, which is responsible for the simulation and GA, is written in C++. This was done for speed advantage over other candidate languages, including C#, Visual Basic, JavaScript. Given the information provided in Table 7. below, C++ was considered the best option as the development times for C and Rust were too taxing. The primary program relied heavily on a third party library called Super Fast Media Library (SFML), which is a graphics library used to draw the robotic simulation. SFML was chosen due to it’s flexibility and portability. The primary program also used OpenMP for multithreading, explained more in 7.2 Parallel Processing.

Table 7.1 - Programming Languages Energy, Time and Memory usage, Source: [26]

42

An online program, written in PHP, is used to distribute workload, and collate results. A third-party framework called Laravel, is used to shortcut development time. These were selected due to having previous experience with these web technologies.

7.2 Parallel Processing The simulation runs each walker on a different thread during execution to significantly increase computation speed [3].

OpenMP is used to divide up the tasks between different threads. There is a primary loop in the code which triggers the execution of the physics for one millisecond for each robot. This entire execution is done on one thread per robot as each part of the robot can potentially affect every other part in the elastic push/pull. The threads are divided at a higher level and reused inside for this loop.

#pragma omp parallel num_threads(16) { // execute 200ms of "subticks" using OpenMP multithreading for (int subticks = 0; subticks < subticks_max; subticks++) { #pragma omp for for (int i = 0; i < robots.size(); i++) { robots[i]->tick(_obstacle); } } } Figure 7.1 - Multithread Loop

“Subticks” here are used to allow the calculation of many ticks (one millisecond of physics) to occur without requiring the creation and destruction of a thread for every single tick.

For more information see Appendix B – Simulation.cpp (page 84)

43

7.3 Distributed Computing To further increase testing speed, a distributed computing network was set up. As shown in Figure 7.2, this was achieved using a custom communication protocol where each worker ran the simulation and GA with a given set of parameters until completion. Once the simulation and GA had finished the prescribed number of generations, the worker would submit the results to a central server and request a new set of testing parameters. This distributed computing allowed a four-fold increase in processing speed [3].

This included advantages such as: •

continuous storage of results,



on-the-fly analytics,



re-prioritisation of testing parameters.

Figure 7.2 - Distributed Computing Model

44

7.3.1 Dispatching Desired test parameters are stored in the database and when a worker requests a new set of parameters to test, the quantity of results (and pending results) for each set of parameters is counted and the set with the least results is dispatched. If there are multiple with the same minimum number, then one is selected at random. In this process, each test eventually gets equal attention and if a new test is added later it will get all the attention for a short period of time. This is also convenient for the case of a faulty or redesigned test, where the results are deem invalid and the test needs to be re-run, that test will get all the attention of the workers until it catches up.

A pending result (mentioned above), occurs when a test has been dispatched but the results haven’t been received yet. This is recorded so the same test isn’t worked on by multiple workers at the same time while other tests have equal or lower result counts.

In the event a worker fails to return the results of an experiment within an hour, the pending result is deleted and the test is put back in the queue for other workers. If the test is then returned after that time it is logged in the system log and an over due test, and the results are dropped.

To see the implementation of this dispatching in code, refer to Appendix K – ApiController.php (page 110).

To see the code responsible for the workers communicating with the server, refer to Appendix I – Communicator.cpp (page 104).

45

7.3.2 Timing and Benchmarking All tests are timed by the worker in milliseconds, however the different workers all have different hardware and different background processes running. These differences create different performance capabilities. This is an issue for analysing time efficiencies. To combat this issue, benchmarking is performed every 50 tests or 6 hours, whichever comes first. When a time analysis is performed, all time is calculated based on relative quantities of benchmark.

When a benchmark command is received, the worker program has the same set of parameters with which to run a test.

The code for this can be seen in Appendix H – Parameters.cpp (page 103)

46

8.0 GA RESULTS 8.1 Fitness Distribution This is an analysis of the distribution of possible fitness values. This also compares the effect of steepest descent vs normal search.

These graphs were generated by performing 1500 tests per chart and scaling the results to be a percentage of the total number.

Figure 8.1 - Fitness Distribution without Steepest Descent

47

Figure 8.2 - Fitness Distribution with Steepest Descent

The effect of steepest descent on the simulation is to prevent many of the test from progressing pas the early stages of the development, mainly preventing the development of locomotion. This is likely due to the local maxima effect, and the effect where steepest descent aggressively tracks towards a local maxima and cannot step backwards to achieve a higher maxima elsewhere.

48

8.2 Effects of Population Size

Figure 8.3 - Graph of Fitness vs Generations for various Population Sizes

Population vs Generations Generations Population

Steepest Descent

20

Off

100

Off

200

Off

mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

57 14 125 20 176 31

93 25 194 67 332 117

72 16 250 82 477 256

108 24 250 109 530 275

127 37 367 153 584 237

134 35 381 89 619 215

132 32 367 134 777 278

163 37 469 249 641 275

138 37 610 326 430 141

145 31 456 175 909 434

Table 8.1 - Population vs Generations Results

These results show the simulation, GA, and distributed processing are working as expected. There is a general trend of larger population and more generations yielding better results which is an expected result for GA.

The benefit of using a population of 200 is statistically significant.

49

8.3 Effects of Duration Variance

Figure 8.4 - Graph of Fitness vs Generations for Duration Variance

This graph shows the fitness output of linear duration variance is not negatively impacted in a statistically significant way and the concept is a sound adjustment to the simulation.

Figure 8.5 - Graph of Runtime vs Generations for Duration Variance

This graph shows the optimisation benefits of linear duration variance over static. The quantity on the y-axis is the quantities of benchmark value described in 7.3.2 Timing and Benchmarking, page 46. This value is derived by dividing the run time by the benchmark run time (a standard simulation run many times per worker). This provides a normalised, consistent evaluation of run time.

50

Figure 8.6 - Graph of Duration Variance: Generations vs Scaled Fitness

This graph can be read as fitness gained per unit time of computation. Scaled Fitness is a value derived by diving the fitness by the duration index. This is a rough heuristic for the fitness gained by increasing computation.

This graph shows diminishing returns for increased computation time (or capacity). The best result is duration variance turned on, without steepest descent, running for only 20 generations.

Duration Variance vs Scaled Fitness Generations Duration Variance

Steepest Descent

Off

Off

On

Off

Off

On

On

On

mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

3809 2040 7369 4728 2854 1405 6915 4204

2715 1794 6211 6236 2717 2381 7070 4089

1976 1754 6204 4759 2808 2125 6972 5742

1997 1308 4556 3499 2015 1906 4621 3840

2877 2153 3278 1835 1874 1766 5111 5474

3058 1477 3582 3109 1690 879 2683 2082

2006 1083 3467 2279 1698 1116 3209 2177

3122 2264 2980 2180 2114 1849 2958 3136

1286 949 3290 2615 2192 1703 2816 2453

2146 1346 2421 2169 1468 810 3072 2298

Table 8.2 - Duration Variance vs Scaled Fitness Results

51

8.4 Mutation Variance 8.4.1 No Mutation Variance Mutation variance the concept where the rate of mutation changes over the course of the generations. This test is a control for comparing different mutation variances. This also compares the effect of steepest descent when compared with different mutation rated.

Figure 8.7 - Graph of Mutation Variance: Generations vs Fitness

52

No Mutation Variance: Mutation Rate vs Fitness Generations Mutation Rate

Steepest Descent

0

Off

1

Off

2

Off

5

Off

10

Off

20

Off

0

On

1

On

2

On

5

On

10

On

20

On

mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

26 5 98 27 114 27 145 39 138 21 33 6 25 5 95 18 124 34 130 25 122 27 85 13

23 4 124 35 177 44 268 57 192 33 51 9 24 5 98 20 150 35 289 112 194 35 115 18

22 5 139 24 166 43 352 121 199 30 50 9 21 3 146 30 180 43 289 107 226 38 157 35

24 4 162 37 189 55 411 176 230 49 48 11 26 3 152 27 186 56 358 122 218 41 195 40

21 4 159 49 239 81 478 196 228 34 42 9 19 4 156 49 211 68 438 145 370 120 176 41

27 6 143 26 257 74 560 215 255 61 45 8 22 5 181 75 342 182 481 199 397 167 185 32

21 5 198 45 318 120 613 296 250 50 46 10 23 4 190 58 236 66 624 310 415 151 258 58

19 4 249 74 305 136 676 282 242 34 48 9 21 4 263 74 279 176 481 239 437 118 274 38

18 5 282 109 274 88 734 243 227 48 40 7 21 4 240 83 214 54 487 203 523 219 256 66

21 3 210 37 336 132 863 276 310 74 42 10 21 3 207 68 302 87 563 286 514 207 320 109

Table 8.3 - No Mutation Variance: Mutation Rate vs Fitness Results

From the data it can be seen that a mutation rate of 5 without steepest descent yields the best results. At higher mutation rate it’s likely the chromosome becomes too unstable and at lower mutation rates the chromosome doesn’t advance at its full potential.

53

8.4.2 Linear Mutation Variance Linear mutation is a simple effect where the rate of mutation decreases over time, as the generations increase. This chart also compares the effect of steepest descent when paired with different mutation rates and linear mutation.

Figure 8.8 - Graph of Linear Mutation Variance: Generation vs Fitness

54

Linear Mutation Variance: Mutation Rate vs Fitness Generations Mutation Rate

Steepest Descent

0

Off

1

Off

2

Off

5

Off

10

Off

20

Off

0

On

1

On

2

On

5

On

10

On

20

On

mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

21 3 125 33 125 27 174 32 121 19 78 16 21 5 112 21 133 26 165 42 127 16 103 22

26 4 165 47 161 40 257 51 203 33 107 25 22 6 150 18 209 62 282 83 185 35 140 20

22 4 164 45 218 52 403 128 288 82 130 20 25 6 179 52 290 77 353 113 290 76 191 38

25 5 271 118 395 158 559 291 475 370 146 29 23 4 240 108 254 68 483 163 521 116 211 45

19 4 260 76 430 209 620 210 686 177 161 29 22 3 359 132 480 157 523 236 620 158 312 93

22 5 288 81 399 205 896 249 820 369 189 33 23 5 359 120 415 183 806 204 599 231 360 65

25 5 324 135 523 243 781 293 1168 220 201 46 25 5 274 98 610 149 798 373 697 228 460 156

23 6 464 188 620 212 920 393 1199 242 191 48 22 3 379 161 464 178 936 240 661 184 488 120

22 3 295 61 619 205 832 345 789 331 215 41 22 4 404 167 413 197 797 301 939 190 473 113

22 4 471 216 633 238 1330 547 1228 275 278 55 21 4 554 288 735 218 1086 504 1071 312 530 179

Table 8.4 - Linear Mutation Variance: Mutation Rate vs Fitness Results

It can be seen from the data that steepest descent reduces the deviation but also reduces the final fitness value. The linear mutation also yield much higher fitness values than the no mutation test.

55

8.4.3 Bitwise Mutation Variance Bitwise mutation is the technique where the most important bits are mutated more in the early generations and less in the later generations, and the reverse is true for least important bits.

Figure 8.9 - Graph of Bitwise Mutation Variance: Generations vs Fitness

56

Bitwise Mutation Variance: Mutation Rate vs Fitness Generations Mutation Rate

Steepest Descent

0

Off

1

Off

2

Off

5

Off

10

Off

20

Off

0

On

1

On

2

On

5

On

10

On

20

On

mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

22 3 87 15 90 17 119 21 132 28 112 17 20 3 74 12 82 16 115 31 136 37 144 22

24 5 89 23 118 30 147 33 240 101 141 25 21 5 94 19 103 28 205 62 205 76 199 48

20 4 99 24 131 23 193 46 301 95 209 44 25 4 97 29 171 53 153 38 233 72 234 36

23 4 121 42 154 32 243 87 329 129 250 49 22 3 151 48 148 42 200 58 293 103 337 118

21 4 149 34 200 72 331 138 488 224 287 95 25 5 109 29 193 51 264 95 463 241 296 87

19 4 119 24 183 43 340 128 637 297 313 92 22 4 128 39 199 56 365 171 392 112 359 84

26 6 147 28 207 63 336 149 505 235 322 109 21 4 123 30 155 24 400 184 418 175 410 130

18 3 149 17 174 42 288 94 635 244 405 122 20 4 142 35 211 82 365 147 516 224 481 153

23 6 177 47 181 40 459 173 668 254 380 92 22 4 159 40 175 50 376 121 423 182 552 171

22 4 185 84 228 72 506 253 1008 400 501 137 24 5 139 40 245 77 430 244 507 220 657 202

Table 8.5 - Bitwise Mutation Variance: Mutation Rate vs Fitness Results

The result of this experiment shows that a mutation rate of 10 is best suited to bitwise mutation, however, linear variance performs better than bitwise.

57

8.5 Selection Pressure The quantities of selection pressure are used to increase the chances of the better performing chromosomes being selected for the next generation. The fitness value is raised to the power of the selection pressure before selection.

Figure 8.10 - Graph of Select Pressure: Generations vs Fitness

58

Selection Pressure vs Fitness Generations Selection Pressure

Steepest Descent

1

Off

2

Off

3

Off

4

Off

1

On

2

On

3

On

4

On

mean var mean var mean var mean var mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

125 19 126 26 123 23 144 34 136 39 125 20 138 28 130 37

251 92 191 46 215 58 204 53 163 40 194 67 240 66 219 56

410 186 229 60 256 91 273 98 275 89 250 82 232 68 268 88

332 114 269 91 329 121 248 80 410 148 250 109 293 65 257 81

443 272 451 252 323 138 382 120 398 153 367 153 372 140 434 144

416 171 521 168 418 167 364 117 270 77 381 89 370 144 326 116

529 221 451 172 321 93 341 113 319 154 367 134 440 182 321 114

443 163 518 208 417 160 633 313 369 139 469 249 656 286 503 190

776 292 449 155 556 189 498 167 475 186 610 326 604 191 395 169

566 240 724 287 390 141 486 209 684 207 456 175 333 99 398 160

Table 8.6 - Selection Pressure vs Fitness Results

These results show that a lower select pressure yields better results, but the difference is not statistically significant.

59

8.6 Crossover Slicing Crossover slicing is the number of times the chromosome is sliced during the crossover process. A slicing value of zero causes no crossover to happen at all, and all chromosomes go straight to the mutation step of the process.

Figure 8.11 - Graph of Crossover Slicing: Generations vs Fitness

60

Crossover Slicing vs Fitness Generations Crossover Slicing

Steepest Descent

0

Off

1

Off

2

Off

5

Off

10

Off

20

Off

0

On

1

On

2

On

5

On

10

On

20

On

mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var mean var

20

40

60

80

100

120

140

160

180

200

114 27 131 16 129 24 129 35 132 25 160 39 137 34 123 27 134 26 137 31 112 25 139 32

220 46 194 64 202 70 254 105 223 51 225 76 216 63 172 40 203 58 169 51 201 46 249 367

243 66 271 69 301 120 266 83 313 130 288 104 261 76 231 76 280 86 286 116 261 62 281 104

286 70 318 97 347 146 365 180 384 280 431 194 274 91 241 41 252 91 266 105 189 43 361 131

353 116 544 216 386 185 329 173 353 165 502 203 371 145 404 136 330 112 377 143 349 148 254 83

477 206 373 152 521 307 432 172 389 161 368 162 383 424 368 117 378 150 416 158 522 268 353 125

647 395 386 134 478 176 590 186 454 213 415 159 363 118 323 122 466 161 457 135 411 170 452 172

387 127 385 152 485 214 658 291 407 166 569 213 356 154 450 178 422 146 501 224 443 156 368 113

563 192 555 220 384 152 555 191 617 165 701 194 549 194 372 156 419 139 491 172 561 260 430 146

588 216 417 181 825 323 602 242 476 217 525 199 546 228 569 249 536 214 511 280 532 186 679 287

Table 8.7 - Crossover Slicing vs Fitness Results

From the results, it can be seen that crossover has very little effect on the final fitness value.

61

9.0 ROBOTIC RESULTS 9.1 Fitness Calculation To validate the results of the experiment, the best configurations of each test were combined and algorithm the was run. However, one simulation also included a ramp obstacle to verify the algorithm could effectively create robots to handle the obstacle.

The samples shown in this results section are images representing the design and progress of the best performing robot in a given generation. These were captured by running the entire GA process 5 times for each fitness calculation, then taking the median result and reanimating the chromosomes from different generations.

62

9.1.1 Distance Travelled This fitness evaluation uses the equation below. This is a value which only accounts for horizontal movement of the robots. For more information on this fitness see 6.2.1 Distance Travelled (page 38).

where

n : total number of nodes in robot node[i].x : x position of node i Equation 9.1 - Distance Travelled Fitness

Figure 9.1 - Distance Travelled, Generation 1

Figure 9.2 - Distance Travelled, Generation 50

63

Figure 9.3 - Distance Travelled, Generation 100

Figure 9.4 - Distance Travelled, Generation 150

Figure 9.5 - Distance Travelled, Generation 200

64

9.1.2 Height by Distance Travelled This is calculated using the equation below. For more information on this fitness, see 6.2.2 Height and Distance Travelled (page 39).

where

n : total number of nodes in robot node[i].x : x position of node i node[i].y : y position of node i Equation 9.2 - Final Height and Distance Fitness

Figure 9.6 - Height by Distance Travelled, Generation 1

Figure 9.7 - Height by Distance Travelled, Generation 50

65

Figure 9.8 - Height by Distance Travelled, Generation 100

Figure 9.9 - Height by Distance Travelled, Generation 150

Figure 9.10 - Height by Distance Travelled, Generation 200

66

9.1.3 Time Averaged Height by Distance Travelled This fitness is calculated using the equation below. For more information on this calculation see 6.2.3 Time Average Height and Distance (page 40).

where

m : total milliseconds of simulation n : total number of nodes in robot node[i][t].x : x position of node i at time t node[i][t].y : y position of node i at time t Equation 9.3 - Fitness Calculation

Figure 9.11 - Time Averaged Height by Distance Travelled, Generation 1

Figure 9.12 - Time Averaged Height by Distance Travelled, Generation 50

67

Figure 9.13 - Time Averaged Height by Distance Travelled, Generation 100

Figure 9.14 - Time Averaged Height by Distance Travelled, Generation 150

Figure 9.15 - Time Averaged Height by Distance Travelled, Generation 200

68

9.2 Ramp Obstacle This ramp obstacle shows robots designed in the simulation with randomised physics are capable of tackling unexpected obstacles they’ve never encountered before. This is an effective demonstration of domain randomisation.

The robots here are the final generation after the running using the domain randomisation technique in fitness evaluation for 200 generations. These robots were then tested against a ramp obstacle and can be seen moving up the ramp.

Figure 9.16 - Population of Robots attempting ramp obstacle

69

10.0 DISCUSSION 10.1 Crossover Slicing Varying the amount of crossover slices from zero to six had no statistically significant impact on the final fitness value. This is a surprising result as some impact was expected.

The effect of increasing the slicing quantity should be to converge two chromosomes (and over time, the whole population), at the possible detriment of the quality of the parent chromosomes. This detriment should come from the destruction of critical structures in the chromosome.

If varying the slicing quantity has no effect, then it’s reasonable to remove this step from the final software product and have a zero crossover.

10.2 Selection Pressure Varying the selection pressure also has very little effect on the final fitness value. The best performing selection pressure is a value of one, which is the same effect as ignoring selection pressure. This means selection pressure can also be safely ignored for the final software product.

70

10.3 Mutation Variance Varying the rate of mutation had a significant impact on the final fitness. By far the most successful variance method is the linear variance. This method simply decreases the rate of mutation as the generations increase.

Bitwise mutation did not perform as well as expected. This fine-tuning of the mutation operator was expected to yield an improvement in overall fitness. [15] The difference in expectation versus measured result could be due to the size of the bit segments. In the original paper on this type of mutation the number of bits being varied was much larger, in this experiment the range of bit to be scaled across was four. The chromosome is broken down into 20 parts, 16 bits each, these parts and broken down again into 4 bits each. It’s these final four bits which had this bitwise effect applied across them, which could have caused the technique to be counterproductive.

10.4 Duration Variance Duration variance showed some impact on final fitness value. However, when duration of tests is taken into account, it has some promising results. This technique is intended to reduce the time taken to achieve a good result. As Figure 8.6 - Graph of Duration Variance: Generations vs Scaled Fitness (page 51) shows, the return on time spent processing is better with duration variance.

This optimisation of running the simulation only for a short time for the first few generations then slowly increasing the duration; shows enough potential to be included in the final software product as an option for development.

71

10.5 Steepest Descent The steepest descent technique did not perform as expected. In many cases it produced worse results than having the optimisation turned off, however, is was not of statistical significance. This technique should be removed from the final software product.

10.6 Domain Randomisation Domain randomisation proved to be effective in producing robots which can handle a wider range of physical simulations. This technique should be included in the final software product, however more investigation may be required where physical prototypes are being produced.

10.7 Cross-Cutting Parameters The results of this project have yielded an interesting nested optimisation problem. The set of testing parameters which are varied above create their own higher-dimensional search space and search this space is very laborious. In several of the tests, the results were skewed by the fix parameters of the test. For example, the mutation variance test showed the base mutation rate has a big impact on results, hiding the effect of variance.

It is observed however, that the different parameters are somewhat independent, and the biggest effect they have on each other is to hide the effect of other parameters when they’re behaving in a destructive manner.

72

10.8 Developmental Stages It has been observed during the experiments that the development of the robots mostly happens in the same three distinct stages; become stable, discover movement, balance while moving. The robots initially fall over quite a bit then they become stable. Usually several generations will pass before much else happens, then they will start to move. This is why the word “discover” was used above. It’s like the robots are trying to work out what to do, then one of them cracks it and the rest all suddenly start doing it too. From there they become un balanced and fall over quite easily, but after a while they become very good a balancing.

73

11.0 CONCLUSION Genetic algorithms (GAs) are an effective way to design these unconventional robots. This is largely aided because the robots operate under a simple set of rules which, when combined can produce complex systems. The construction of the chromosome allowed the very structure of the robots to be chopped and changed very quickly, imposing very few rules on the simulation or algorithm. The end result is the algorithm had a lot of flexibility to express the simple rules of it’s building materials to achieve its goals.

Distributing the work load across several computers achieved a level of processing power not possible on the given budget via conventional processing. This greatly assisted in creating quality data and yielded very interesting designs of the robots. This technique also created a very useful process for health checks, redundancy, robust collection of data and convenient uptime management.

Some of the final designs of the robots were very unexpected and novel. This algorithm went a long way outside the conventional structure and appearance of robots while still yielding attractive performance metrics, including land speed ability. These robots, any others produced by this algorithm in the future, could have the ability to inspire designs in the real world.

Soft robotics and Tensegrity robotics are an emerging field with exciting possibilities. The inherent structure and design of these robots lend them very readily to GAs for optimisation. The emergent complexities of this combination of technologies have the potential to yield very advanced functionality, making big leaps forward with the use and application of robots for solving a wide range of problems.

74

The results found in this experiment can be used to modify the worker simulation software to create a piece of software which will rapidly produce 2D or 3D designs for robots in varying conditions. This software could be used by NASA or others to produce designs or inspiration for designs by creating novel solutions to difficult terrains. The use of GAs in this work has proved to be an effective means of finding un expected but effective designs which could be of great benefit to robotic designers in the future.

75

12.0 FUTURE WORK 12.1 Enhance Robotic Simulation The robotic simulation used here only represents robots in 2 dimensions, however for a more use approach, the simulation will need to be extended to 3 dimensions. This 2D simulation was sufficient to assess the chromosomes for their fitness and run the GA, and validate the results. However to truly design robots for industry, a third dimension must be added.

The simulation could also make changes to the building materials available to the algorithm. For example, the concept of a “cable”, could be added. This would bring the design more in line with the robots being designed by the University of California Berkeley [1].

The robots could also be given the ability to attach muscles mid-way along other muscles. This would create much stronger lever action and could potentially radically alter the end result. This would be a move away from tensegrity, but it may assist in some very specific purposes.

12.2 Effect of Domain Randomisation The results of domain randomisation in this report were promising, however further investigation is needed. After the above improvements are made (enhancing the robotic simulation and introducing the third dimension), and after physical prototypes are built based on designs from this software, the true effect of domain randomisation should be tested.

76

12.2 Neural Network A concept which could prove interesting is the idea developing a neural network and train it to improve given chromosomes. As illustrated below, a network could take a chromosome as an input and output a slightly (or significantly) improved chromosome.

Figure 12.1 - Robotic Enhancement Neural Network

The reward/punishment of the network would be derived by simulating both the input and output chromosomes; rewarding the network if the output chromosome has a better fitness than the input chromosome and punishing it if the output performs worse.

Over time this process should yield a network which improves chromosomes. As the network becomes reliable at improving chromosomes, it may plateau and a threshold for improvement may be required for continued gains from training. If a certain threshold of improvement is not met, then the trial would be considered a failure.

77

In this way, there is theoretically limitless improvement until the network overfits the data and continuously outputs the same one or two chromosomes [27]. However, it’s very likely that these would be at or near the global maximum. The likelihood of the network being stuck at a local maximum or saddle is very low given the continuously random input forcing a chaotic, simulated annealing style optimization. [27]

78

13.0 REFERENCES

[1] “About Squishy Robotics,” University of California, Berkley, [Online]. Available: https://squishy-robotics.com/about.html. [Accessed 14 08 2018].

[2] J. F. Stoddart, “From Supramolecular to Systems Chemistry: Complexity Emerging out of Simplicity”.

[3] J. Masters, “Progress Report: Distributed Evolutionary Computation for Robotic Design in Balancing and Movement,” Perth, 2018.

[4] A. Agogino, V. SunSpiral and D. Atkinson, “Super Ball Bot - Structures for Planetary Landing and,” NASA Ames Research Center, 2013.

[5] G. M. Whitesides, “Soft Robotics”.

[6] M. Neville and A. Sibley, “Developing a Generic Genetic Algorithm,” 2002.

[7] J. Cagan and A. M. Agogino, “Innovative design of mechanical structures from first principles,” vol. 1, no. 3, 2009.

[8] E. W. Aslaksen, “System Thermodynamics:A Model IllustratingComplexity Emerging fromSimplicity,” Sinclair Knight Merz, pp. 283-284, 2004.

79

[9] “Soft-Robotic Rover with Electrodynamic Power Scavenging NIAC Phase 1 Final Report,” Cornell University, Ithaca, New York, 2016.

[10] N. F. Michelena and A. M. Agogino, “Multiobjective Hydraulic Cylinder Design”.

[11] J. W. Kim , I. M. Van Meerbeek , B. C. Mac Murray , S. S. Robinson , P. X. Zou , M. N. Silberstein and R. F. Shepherd, “Morphing Metal and Elastomer Bicontinuous Foams for Reversible Stiffness, Shape Memory, and Self‐Healing Soft Machines,” Advanced Materials, vol. 28, no. 14, pp. 2801-2806, 2016.

[12] M. Andrychowicz, B. Marcin, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng and W. Zaremba, “Learning Dexterous In-Hand Manipulation,” OpenAI, 2018.

[13] B. C. Stadie, S. Levine and P. Abbeel, “Incentivizing exploration in reinforcement learning with deep predictive models,” in ICLR, Berkeley, 2016.

[14] J. Holland, Adaptation in Natural and Artificial Systems, Michigan: University of Michigan, 1975.

[15] C. T. Fogarty, K. Y. Chan and M. E. Aydin, “Main effect fine-tuning of the mutation operator and the,” Springer-Verlag, 2006.

[16] M. E. Requena-Pérez, A. Albero-Ortiz, J. Monzó-Cabrera and A. Díaz-Morcillo, “Combined Use of Genetic Algorithms and Gradient Descent Optmization Methods

80

for Accurate Inverse Permittivity Measurement,” IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, vol. 54, no. 2, 2006.

[17] C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning), Cambridge, Massachusetts: The MIT Press, 2006.

[18] T. Karras, T. Aila, S. Laine and J. Lehtinen, Progressive growing of GANs for improved quality, stability, and variation.

[19] J. Oh, J. Baik and S.-H. Lim, “A Model Independent S/W Framework for Search-Based Software Testing,”

e Scientific World Journal, vol. 2014, 2014.

[20] T. V. Mathew, “Genetic Algorithm,” Indian Institute of Technology Bombay, Mumbai, 2002.

[21] A. M. Connor and K. Shea, “A COMPARISON OF SEMI-DETERMINISTIC AND STOCHASTIC SEARCH TECHNIQUES,” Springer-Verlag, London, 2000.

[22] M. Angelova and T. Pencheva, “Tuning Genetic Algorithm Parameters to Improve Convergence Time,” International Journal of Chemical Engineering, vol. 2011, 2011.

[23] H. Knight, “Squishy robots: Phase-changing material could allow even low-cost robots to switch between hard and soft states.,” MIT, 14 July 2014. [Online]. Available: http://news.mit.edu/2014/squishy-robots-0714.

81

[24] T. E. Flemons and D. Blostien, “New Approaches to Mechanizing Tensegrity Structures,” 2017.

[25] L. M. Rocha, “Contextual Genetic Algorithms: Evolving Developmental Rules,” in Advances in Artificial Lif, Los Alamos, Los Alamos National Laboratory, 1995, pp. 368-382.

[26] J. Elizabeth, “Java is one of the most energy-efficient languages, Python among least energy

efficient,”

Jax

Enter,

18

September

2017.

[Online].

Available:

https://jaxenter.com/energy-efficient-programming-languages-137264.html. [Accessed 27 August 2018].

[27] Y. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli and Y. Bengio, “Identifying and attacking the saddle point problem in high-dimensional non-convex optimization,” 2014.

82

APPENDIX A – UPDATED PLAN

83

APPENDIX B – SIMULATION.CPP #include "stdafx.h" #include #include using namespace std; result * multiGenerations(renderer* rendererz, parameters * params) { srand(time(NULL)); // Use no obstacle by default obstacle * _obstacle = new obstacle(0); // Set up genetic algorithm ga * _ga = new ga( params->population, 20, params->selection_pressure, params->mutation_rate, params->mutation_variance, params->steepest_decent, params->crossover_rate ); // clock_t c_start, c_draw, c_reset; c_start = clock(); c_draw = clock(); c_reset = clock(); double total_ticks = 0; int resets = 0; float average_rate = 1; float seconds; long max_fittness; string best_chromosome; int max_generations = params->generations; // Initialize population with random chromosomes std::vector robots; std::vector population; population = _ga->newGeneration(); for (int generation = 1; generation generations; #pragma omp for for (int i = 0; i < population.size(); i++) { robot * r = new robot(population[i], false); if (r->alive) { #pragma omp critical robots.push_back(r); } else { delete r; } }

84

// Calculate simulation run time int sim_duration = params->duration; if (params->duration_variance == 1) { // linearly increase the duration time sim_duration = params->duration * gen_ratio + 1; } // Run simulation on population for entire duration for (int ticks = 0; ticks < sim_duration; ) { int subticks_max = 200; #pragma omp parallel num_threads(16) { // execute 200ms of "subticks" using OpenMP multithreading for (int subticks = 0; subticks < subticks_max; subticks++) { #pragma omp for for (int i = 0; i < robots.size(); i++) { robots[i]->tick(_obstacle); } } } ticks += subticks_max; total_ticks += subticks_max; // redraw the animation evey 50ms seconds = (float)(clock() - c_draw) / (float)CLOCKS_PER_SEC; if (seconds > 0.05 && robots.size() > 0) { c_draw = clock(); rendererz->update(&robots, _obstacle); } // Abort loop if entire population is dead if (robots.size() == 0) { break; } } // Perform analysis after simulation if (robots.size() > 0) { max_fittness = -1; double fittness = -1; robot* fittest = robots[0]; for (int i = 0; i < robots.size(); i++) { fittness = robots[i]->fittness(); if (fittness > max_fittness) { fittest = robots[i]; best_chromosome = robots[i]->gene->toString(); max_fittness = fittness; } } seconds = (clock() - c_start) / CLOCKS_PER_SEC; float avg = total_ticks / seconds; if (average_rate == -1 || isinf(average_rate)) { average_rate = avg; } else { average_rate = (average_rate + avg) / 2; }

85

cout id, _result->chromosome, _result>fitness, _result->millis); } _communicator->pushResults(params->id, _result->fitness, _result->millis); delete _result; delete params; } } int main(int argc, const char * argv[]) { renderer* _renderer = new renderer(700, 1400); if (argc == 1) { runDistributedSim(_renderer); } else if (argc == 2 && argv[1] == "debug") { parameters * params = new parameters(); multiGenerations(_renderer, params); delete params; } }

87

APPENDIX C – GA.CPP #pragma once #include "stdafx.h" #include

ga::ga(int size, int length, int selectionPressure, int mutationRate, int mutationVariance, bool steepestDecent, int crossover) : generator(rd()), distribution(0, LONG_MAX) { _rndCore = 0; _size = size; _length = length; _selectionPressure = selectionPressure; _mutationRate = mutationRate; _mutationVariance = mutationVariance; _steepestDecent = steepestDecent; _crossover_rate = crossover; _temp.reserve(_size); } /* Generate a random long less then max */ long ga::randLONG(long max) { return (distribution(generator) % (max + 1)); } /* Generate a random long long less than max */ long long int ga::randLLONG(long long int max) { long long int r1 = distribution(generator); long long int r2 = distribution(generator); long long int r3 = distribution(generator); return ((r1 * r2 * r3) % (max + 1)); } /* Crossover two chromosomes and insert them back into the population */ void ga::crossover(std::vector* a, std::vector* b, std::vector* population, int i) { // initialize random time srand(time(NULL)); bool slice = randLONG(1) == 0; std::vector c; std::vector d; c.reserve(_length); d.reserve(_length); // if zero crossover, copy the first chromosome verbatim if (_crossover_rate reserve(_length); for (int j = 0; j < _length; j++) { dna->push_back(randLONG(65536)); } return dna; } /* Create an entirely new population */ std::vector ga::newGeneration() { std::vector population; population.reserve(_size); for (int i = 0; i < _size; i++) { chromosome * c = new chromosome(*newDna()); robot * r = new robot(c, false); if (!r->alive) { delete c; i--; } else {

89

population.push_back(c); } delete r; } return population; } /* Choose a chromosome based on the roulette wheel */ std::vector* ga::choose(long long int max) { long long int v = randLLONG(max); for (int i = 0; i < _size; i++) { v -= _temp[i]->weighted_rank; if (v < 0) { return &_temp[i]->dna; } } return newDna(); } /* Sort evaluation */ bool sortGene(chromosome* a, chromosome* b) { return (a->fittness > b->fittness); } /* Breed the population */ void ga::breed(std::vector* population) { std::sort(population->begin(), population->end(), sortGene); for (int i = 0; i < _size; i++) { _temp.push_back(population[0][i]); } srand(time(NULL)); // Handle very large values for fitness (becomes an issue when selection pressure == 4) long long int total = 0; #pragma omp parallel for reduction(+:total) for (int i = 0; i < _size; i++) { long long int weighted = _temp[i]->fittness; if (weighted > 0) { weighted = pow(weighted, _selectionPressure); total += weighted; _temp[i]->weighted_rank = weighted; } else { _temp[i]->weighted_rank = 0; } } if (_steepestDecent) { // Copy the top performing chromosome into the next generation population[0][0] = _temp[0]; } #pragma omp for for (int i = (_steepestDecent ? 1 : 0); i < _size; i+= 2) { std::vector* a = choose(total); std::vector* b = choose(total); crossover(a, b, population, i); } #pragma omp for for (int i = (_steepestDecent ? 1 : 0); i < population->size(); i++) { delete _temp[i];

90

} _temp.clear(); } /* Calculation the bitwise mutation scaling */ float ga::bitwiseRate(float gen_ratio, int bit_significance) { float y = gen_ratio; float x = 3-bit_significance; float bit_ratio = (-2.0 / 3.0)*x - y + (4.0 / 6.0)*x*y + 2.0; return _size / (_mutationRate * bit_ratio); } /* Get random bits for mutation */ uint16_t ga::randBits(float gen_ratio) { uint16_t retval = 0x00; switch (_mutationVariance) { case 0: // No mutation variance for (int i = 0; i < 16; i++) { if (randLONG(_size / _mutationRate + 1) == 0) { retval += pow(2, i); } } break; case 1: // Linear mutation variance for (int i = 0; i < 16; i++) { if (randLONG(_size / (_mutationRate / gen_ratio) + 1) == 0) { retval += pow(2, i); } } break; case 2: // Bitwise mutation variance for (int a = 0; a < 4; a++) { for (int bit = 0; bit < 4; bit++) { int i = a * 4 + bit; if (randLONG(bitwiseRate(gen_ratio, bit)) == 0) { retval += pow(2, i); } } } break; } return retval; } /* Mutate an entire population */ void ga::mutate(std::vector * population, float gen_ratio) { if (_mutationRate == 0) { return; } #pragma omp for for (int i = (_steepestDecent ? 1 : 0); i < _size; i++) { std::vector * dna = &population[0][i]->dna; for (int j = 0; j < _length; j++) { dna[0][j] = dna[0][j] ^ randBits(gen_ratio * 2); } } }

91

/* Get size */ int ga::size() { return _size; }

92

APPENDIX D – ROBOT.CPP #pragma once #include "stdafx.h" float randFloat(float min, float max) { return min + static_cast (rand()) / (static_cast (RAND_MAX / (max - min))); } robot::robot(chromosome * _gene, bool randomisedPhysics) { gene = _gene; alive = false; gene->fittness = 0; _gravity = 9800; _airFriction = 0.002; _groundFriction = 2; _groundHardness = 100; _muscleElasticity = 2; _inertia = 2; if (randomisedPhysics) { _gravity *= randFloat(0.1, 2); _airFriction *= randFloat(0.1, 10); _groundFriction *= randFloat(0.1, 2); _groundHardness *= randFloat(0.1, 10); _muscleElasticity *= randFloat(0.1, 2); _inertia *= randFloat(0.1, 2); } // Create joints from chromosome for (int i = 0; i < gene->dna.size(); i++) { uint8_t * data = gene->get(i); uint8_t t = data[0] / 2; if (t >= 0 && t dna.size(); i++) { uint8_t * data = gene->get(i); uint8_t t = data[0] / 2; if (t >= 3 && t a == muscles[i]->b) { // This is an invalid robot // No muscle should join to the same node at both ends return; } } // Set a few select muscles to oscillate based on chromosome for (int i = 0; i < gene->dna.size(); i++) { uint8_t * data = gene->get(i); uint8_t t = data[0] / 2; if (t == 7) { uint8_t a = data[1] * 15; uint8_t b = data[2] * 15; uint8_t c = data[3] * 15; int m_id = (a + i) % muscles.size(); muscles[m_id]->setOsc(b, c - 122); } delete data; } // Slide the whole robot down until at least one joint is touching the ground int lowest_joint = INT_MAX; for (int i = 0; i < joints.size(); i++) { if (joints[i]->position->y < lowest_joint) { lowest_joint = joints[i]->position->y; } } for (int i = 0; i < joints.size(); i++) { joints[i]->position->y -= lowest_joint; }

94

// Successfully constructed robot alive = true; } /* One millisecond tick */ void robot::tick(obstacle * _obstacle) { if (alive) { osc(); reaction(); gravity(); friction(); floor(_obstacle); applyForce(); momentum(); fittness(); } } /* Calculate running fitness total */ long robot::fittness() { long total = 0; if (alive) { for (int i = 0; i < joints.size(); i++) { total += (joints[i]->position->x * joints[i]->position->y) ; } } if (total fittness = 0; alive = false; return 0; } else { gene->fittness = total / joints.size(); } return gene->fittness; } /* Calculate elastic reaction force */ double robot::springForce(float sd) { float q = pow(sd / 4, _muscleElasticity); if (sd < 0) { q *= -1; } return sd * 4 + q; } /* Apply oscillation to any muscles with it turned on */ void robot::osc() { for (int i = 0; i < muscles.size(); i++) { if (muscles[i]->osc_speed != 0) { muscles[i]->oscTick(); } } } /* Calculate reaction forces of the joints from the muscles */ void robot::reaction() { float desiredlength, delta, scaled, rX, rY, accY, accX; double length, force;

95

for (int i = 0; i < muscles.size(); i++) { muscle* m = muscles[i]; length = m->length(); desiredlength = m->desiredLength(); delta = length - desiredlength; scaled = m->strength * delta; force = springForce(scaled); force = force / TICK_PER_SEC; rX = m->dX() / length; rY = m->dY() / length; accY = rY * force; accX = rX * force; m->a->force->x -= accX; m->a->force->y -= accY; m->b->force->x += accX; m->b->force->y += accY; } } /* Apply gravity */ void robot::gravity() { for (int i = 0; i < joints.size(); i++) { joints[i]->velocity->y -= _gravity / TICK_PER_SEC; } } /* Apply air friction */ void robot::friction() { for (int i = 0; i < joints.size(); i++) { joints[i]->velocity->x *= (1 - _airFriction); joints[i]->velocity->y *= (1 - _airFriction); } } /* Apply calculated forces to velocity */ void robot::applyForce() { for (int i = 0; i < joints.size(); i++) { joints[i]->velocity->x += joints[i]->force->x / (joints[i]->weight * _inertia); joints[i]->velocity->y += joints[i]->force->y / (joints[i]->weight * _inertia); joints[i]->force->x = 0; joints[i]->force->y = 0; } } /* Apply calculated velocity to position */ void robot::momentum() { for (int i = 0; i < joints.size(); i++) { joints[i]->position->x += joints[i]->velocity->x / TICK_PER_SEC; joints[i]->position->y += joints[i]->velocity->y / TICK_PER_SEC; } } /* Find angled reaction and friction forces from obstacle shape */

96

float pi = 3.14159265359; float _2pi = pi * 2; float pi_2 = pi / 2; float pi_8 = pi / 8; point * findComponent(point * velocity, float c_ang) { float v_ang = velocity->angle(); float v_mag = velocity->magnitude(); float c_mag = cos(v_ang - c_ang) * v_mag; return new point(c_mag * cos(c_ang), c_mag * sin(c_ang)); } void robot::floor(obstacle * _obstacle) { float impact; float friction; for (int i = 0; i < joints.size(); i++) { edge * e = _obstacle->edgeAt(joints[i]->position->x); if (joints[i]->position->y y) { point * impact_velocity = findComponent(joints[i]->velocity, e->angle - pi_2); point * impact_force = findComponent(joints[i]->force, e->angle - pi_2); point * tangent_velocity = findComponent(joints[i]->velocity, e->angle); point * tangent_force = findComponent(joints[i]->force, e->angle); float deceleration_interval = 1000 / _groundHardness; // default 10 ms float friction = ((impact_velocity->magnitude() / deceleration_interval) * (float)joints[i]->weight + impact_force->magnitude()) * _groundFriction; float tangent_magnitude = tangent_velocity->magnitude() * (joints[i]->weight * 2) + tangent_force->magnitude(); // Static friction if (tangent_magnitude < friction) { joints[i]->force->y -= (tangent_velocity->y * (joints[i]->weight * 2) + tangent_force->y); joints[i]->force->x -= (tangent_velocity->x * (joints[i]->weight * 2) + tangent_force->x); } else { joints[i]->force->y -= (tangent_velocity->y * (joints[i]->weight * 2) + tangent_force->y) / 50; joints[i]->force->x -= (tangent_velocity->x * (joints[i]->weight * 2) + tangent_force->x) / 50; } joints[i]->force->y = joints[i]->force->y - impact_velocity->y * (joints[i]->weight * 2) - impact_force->y; joints[i]->force->x = joints[i]->force->x - impact_velocity->x * (joints[i]->weight * 2) - impact_force->x; joints[i]->position->y = e->y; delete impact_force; delete impact_velocity; delete tangent_velocity; delete tangent_force; } delete e; } }

97

robot::~robot() { for (int i = 0; i < joints.size(); i++) { delete joints[i]; } joints.clear(); for (int i = 0; i < muscles.size(); i++) { delete muscles[i]; } muscles.clear(); }

98

APPENDIX E – MUSCLE.CPP #pragma once #include "stdafx.h" muscle::muscle(joint* _a, joint* _b, float s) { a = _a; b = _b; _desiredLength = length(); strength = s; osc_speed = 0; osc_range = 0; } bool muscle::hasOsc() { return osc_speed != 0; } void muscle::setOsc(float _osc_range, float _osc_speed) { osc_pos = 0; osc_speed = _osc_speed; osc_range = _osc_range; } void muscle::oscTick() { osc_pos += (5 * osc_speed) / TICK_PER_SEC; if (abs(osc_pos) > osc_range) { osc_speed *= -1; } } float muscle::desiredLength() { return _desiredLength + osc_pos; } float muscle::dX() { return a->position->x - b->position->x; } float muscle::dY() { return a->position->y - b->position->y; } double muscle::length() { return sqrt(pow(dX(), 2) + pow(dY(), 2)); } float muscle::angle() { return atan(dY() / dX()); }

99

APPENDIX F – JOINT.CPP #pragma once #include "stdafx.h"

joint::joint(point* pos, int w) { position = pos; velocity = new point(0, 0); weight = w + 30; force = new point(0, 0); } joint::~joint() { delete position; delete velocity; delete force; } bool joint::inf() { return isinf(position->x) || isinf(position->y) || isinf(velocity->x) || isinf(velocity->y) || isinf(force->x) || isinf(force->y); }

100

APPENDIX G – CHROMOSOME.CPP #include "stdafx.h" #include "chromosome.h" using namespace std; uint8_t get_nybble(std::uint16_t number, const unsigned short part) { return (number >> (4 * part)) & 0xF; } chromosome::chromosome(std::vector _dna) { dna = _dna; fittness = 0; } chromosome::chromosome(const char * gene[], int size) : dna() { for (int i = 0; i < size; i+=4) { int sum = 0; for (int j = 0; j < 4; j++) { sum += atoi(gene[i + j]) * (int)pow(16, j); } dna.push_back(sum); } } chromosome::chromosome(string str) : dna() { fittness = 0; int index = 0; while (str.length() > 0) { index = str.find(' '); string num = str.substr(0, index); index++; str = str.substr(index, str.length() - index); dna.push_back(atoi(num.c_str())); } } chromosome * chromosome::clone() { std::vector temp; temp.reserve(dna.size()); for (int i = 0; i < dna.size(); i++) { temp.push_back(dna[i]); } return new chromosome(temp); } uint8_t * chromosome::get(int i) { uint16_t number = dna[i]; uint8_t * retval = new uint8_t[4]; retval[0] = get_nybble(number, 0); retval[1] = get_nybble(number, 1); retval[2] = get_nybble(number, 2); retval[3] = get_nybble(number, 3); return retval; }

101

std::string chromosome::toString() { std::stringstream ss; for (int i = 0; i < dna.size(); i++) { uint8_t * data = get(i); ss mutation_rate = 2; this->mutation_variance = 0; this->duration_variance = 1; this->steepest_decent = true; this->obstacle = 3; } bool parameters::parse(std::string input) { this->complete = false; if (input == "benchmark") { this->complete = true; this->benchmark = true; this->id = "benchmark"; } else { this->benchmark = false; vector strings; istringstream f(input); string s; while (getline(f, s, ',')) { strings.push_back(s); } if (strings.size() < 10) { this->complete = false; return false; } this->complete = true; this->id = strings[0]; this->population = std::stoi(strings[1]); this->generations = std::stoi(strings[2]); this->selection_pressure = std::stoi(strings[3]); this->duration = std::stoi(strings[4]); this->crossover_rate = std::stoi(strings[5]); this->mutation_rate = std::stoi(strings[6]); this->mutation_variance = std::stoi(strings[7]); this->duration_variance = std::stoi(strings[8]); this->steepest_decent = std::stoi(strings[9]); this->obstacle = std::stoi(strings[10]); } return this->complete; }

103

APPENDIX I – COMMUNICATOR.CPP #include "stdafx.h" #include static size_t WriteCallback(void *contents, size_t size, size_t nmemb, void *userp) { ((std::string*)userp)->append((char*)contents, size * nmemb); return size * nmemb; } void gen_random(char *s, const int len) { srand(time(NULL)); static const char alphanum[] = "0123456789" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "abcdefghijklmnopqrstuvwxyz"; for (int i = 0; i < len; ++i) { s[i] = alphanum[rand() % (sizeof(alphanum) - 1)]; } s[len] = 0; } communicator::communicator(string base) : _base(base) { _token = getToken(); } std::string communicator::getToken() { string token; ifstream ifile("token.txt"); if (ifile.good() && ifile.is_open() && getline(ifile, token)) { cout muscles[i]>strength); if (r->muscles[i]->hasOsc()) { drawOscDot(r->muscles[i]->a->position, r->muscles[i]->b->position, r>muscles[i]->osc_speed); } } } void renderer::drawJoint(point * p, int weight) { float radius = 5.f; sf::Color c = sf::Color(0, 255 - weight, 0, 255); sf::CircleShape shape(radius); shape.setFillColor(c); shape.setPosition(modX(p->x) - radius, modY(p->y) - radius); _window.draw(shape); } void renderer::drawMuscle(point * a, point * b, int strength) { sf::Color c = sf::Color(255 - strength, 50, 50, 255); drawLine(a, b, c); } void renderer::drawOscDot(point * a, point * b, int osc_speed) { int x = (a->x + b->x) / 2; int y = (a->y + b->y) / 2; float radius = 4.f; sf::Color c = sf::Color(0, 0, osc_speed * 5, 255); sf::CircleShape shape(radius); shape.setFillColor(c); shape.setPosition(modX(x) - radius, modY(y) - radius); _window.draw(shape); } void renderer::drawLine(point * a, point * b, sf::Color c) { sf::Vertex line[] = { sf::Vertex(sf::Vector2f(modX(a->x), modY(a->y))), sf::Vertex(sf::Vector2f(modX(b->x), modY(b->y))) }; line[0].color = c; line[1].color = c; _window.draw(line, 2, sf::Lines); } void renderer::drawObstacle(obstacle * _obstacle) { std::vector * points = _obstacle->points(); sf::Vertex * line = (sf::Vertex *)malloc(points->size() * sizeof(sf::Vertex)); for (int i = 0; i < points->size(); i++) {

108

line[i] = sf::Vertex(sf::Vector2f(modX(points[0][i]->x), modY(points[0][i]->y))); line[i].color = sf::Color(0, 0, 0, 255); } _window.draw(line, points->size(), sf::Lines); free(line); }

109

APPENDIX K – APICONTROLLER.PHP