Moved on to neuroevolution
I stopped programming back in mid to late 2018. I was too busy with work, I grew tired of dealing with Python, and my main obsession flipped to something else, whatever I ended up doing back then. This time I found the excuse to properly learn Rust through implementing the extremely convoluted logic of one of GMT’s most complicated board games. Turned out that it was that, an excuse: when programming that system stopped being a challenge because I had learned every aspect of Rust it would make sense to apply for that model, I lost interest. Fortunately by then I had become proficient enough in the language that I could move on to my main programming interest: machine learning. I went back to neuroevolution, the art of evolving neural networks through genetic algorithms, something that isn’t remotely as complicated as it sounds. Back then I had implemented it in Python almost from scratch. My concern with Rust is that with the language being so new (although it already feels mature as of 2020) and data scientists focused on exploiting the old, broken beast that is Python, there would be little usable in regards to machine learning for Rust. And I was right to a certain extent, because what seems to be usable uses Python and C++ bindings, or tensors. I don’t understand tensors, they don’t sound good to me in general. So I cobbled together what turned out to be a quite decent implementation of neuroevolution from zero. As usual, the code is available in GitHub: link.
“Crusader Kings 3”, what seems to me the best game since “The Witcher 3”, lingers in my SSD, as I can’t stop thinking about developing more stuff for this neuroevolution shit. I’ve managed to develop a small program to evolve images.
It isn’t much so far, but it works: the main training program will either spit out a generation of images from zero, or load a previous generation (hopefully rated on their individual merits) and produce the next one. Initially I had to open each json file and write the fitness score manually, but it annoyed me enough that I developed a little program to score them through passing the genome identifier and the score as command line parameters. As soon as I develop a small third program to force manually selected genomes to produce their corresponding image to user defined dimensions, I’ll release them on the GitHub page (which, again, is here).
Neuroevolution works like this: you start by needing the computer to decide on some stuff, anything at all. Might be as small as choosing heads or tails, or deciding on where to invest in the stock market. You always have values gathered from some environment, and what values you decide to pass to the computer is up to you. You need to analyze the environment properly, because you might miss vital aspects that the computer would reasonably need to consider to make an appropriate decision. For my generating images model, I settled on very few input parameters. All we know from the environment is the dimensions of the image the computer has to produce, and the aspects of those dimensions are in this case how far the current pixel is from the extremes of the dimensions (top left, top, top right, left, center, right, bottom left, bottom, bottom right). When the computer is fed those calculated values, for each pixel of the image the computer is queried on what RGBA value to put there. That means that for a 256 x 256 image, the computer needs to be queried 65536 times. Some algorithm should rate the proficiency of that final result, and in this case the algorithm is the brain of the human user. Then, an evolutionary algorithm based on genetic programming crosses over the involved genomes, potentially mutates some aspects of them, and then produces a new population that in the next generation will be queried to produce another set of images.
At the core of this process are, of course, neural networks, the almost magical black boxes to which you pass some values which get altered in the neural network’s black innards, only to get spat out through the other side. The internal composition of the neural network, unless you evolve it through an algorithm like NEAT (which I chose not to get into), is decided by the user. It could have as few as a layer and as few neurons as the outputs the network needs to produce (four in this case, red, green, blue and alpha for each pixel). Potentially it can have hundreds of layers and millions of neurons. At the core of the neural network are neurons: the nodes that receive values either coming from outside of the neural network or from the previous layer. The nodes then apply a magical mathematic thing called an activation function, that transmogrifies the input value to some other. The following image represents some activation functions.
Each node has a single activation function (there might be variations on this I’m not aware of), and each of them produces different visible results on the decision. Whether or not they help will depend on how their decision is rated, and whether they get selected out through evolution.
That’s as much of an explanation as I care to give on the subject. In any case, implementing a neuroevolutionary algorithm almost from scratch on Rust took some shit: I had to get deep into generics and closures, which I had feared dealing with through my previous project. To destroy my fears and out of general obsessive-compulsiveness, I followed the Jurassic Park principle: I was so focused in whether or not I could that I didn’t stop to think if I should. Generics help you use potentially infinite variations of each part involved in a process. In this case I didn’t use any variation: you use a single type of population, genome, neural network, layer and node, and yet I made it so I can expand the system at any point of the future. Any cloner of the repository can do so as well (or should be able to anyway). The cost of that is that the client has to write code like this in main:
Which in turn looks something like this in the corresponding structures (this is all the code for training any population, without the intentions of the client code):
The following are the declarations of the types and such for the controller that deals with training a population:
Pictured: the shocking need to learn algebra.
As I was writing all the generic functions, I wondered where the border between concretion and abstraction was. Generics have no runtime performance penalties in Rust: the compiler turns all generics into concrete implementations during compilation. I struggled to understand how to deal with the inevitable points where you just had to create a concrete class (neurons, populations, neural networks), and there’s no way to create something concrete from generics. The solution ended up being obvious, but it required a different mindset: you need to inject the production of concrete structures into the generic frame. Still, how on Earth would you inject something that produces concretion, when the specific details of whatever you need to construct isn’t known most of the time until the structures need to be created?
The solution is closures. You just pass functions as parameters to the generic core. It was obvious, but it’s such a high level, duck-typing-mindset solution used in languages like Python or Javascript that they seemed impossible to implement in such a hardcore language like Rust. But it’s tremendously easy: the type of a function/closure parameter is something like Fn(u32) -> u32, meaning it receives an unsigned integer and spits out an unsigned integer, and as the value of that parameter you could pass something like |value| { value } (or even without the brackets). That’s it. Value gets passed as a parameter whenever the generic code uses the function, and the client doesn’t need to worry about anything else.
That’s about as much as I can think to write right now. I’ll soon release the programs to evolve images, and I’ll likely play “Crusader Kings 3” for a few days.
“Crusader Kings 3”, what seems to me the best game since “The Witcher 3”, lingers in my SSD, as I can’t stop thinking about developing more stuff for this neuroevolution shit. I’ve managed to develop a small program to evolve images.
It isn’t much so far, but it works: the main training program will either spit out a generation of images from zero, or load a previous generation (hopefully rated on their individual merits) and produce the next one. Initially I had to open each json file and write the fitness score manually, but it annoyed me enough that I developed a little program to score them through passing the genome identifier and the score as command line parameters. As soon as I develop a small third program to force manually selected genomes to produce their corresponding image to user defined dimensions, I’ll release them on the GitHub page (which, again, is here).
Neuroevolution works like this: you start by needing the computer to decide on some stuff, anything at all. Might be as small as choosing heads or tails, or deciding on where to invest in the stock market. You always have values gathered from some environment, and what values you decide to pass to the computer is up to you. You need to analyze the environment properly, because you might miss vital aspects that the computer would reasonably need to consider to make an appropriate decision. For my generating images model, I settled on very few input parameters. All we know from the environment is the dimensions of the image the computer has to produce, and the aspects of those dimensions are in this case how far the current pixel is from the extremes of the dimensions (top left, top, top right, left, center, right, bottom left, bottom, bottom right). When the computer is fed those calculated values, for each pixel of the image the computer is queried on what RGBA value to put there. That means that for a 256 x 256 image, the computer needs to be queried 65536 times. Some algorithm should rate the proficiency of that final result, and in this case the algorithm is the brain of the human user. Then, an evolutionary algorithm based on genetic programming crosses over the involved genomes, potentially mutates some aspects of them, and then produces a new population that in the next generation will be queried to produce another set of images.
At the core of this process are, of course, neural networks, the almost magical black boxes to which you pass some values which get altered in the neural network’s black innards, only to get spat out through the other side. The internal composition of the neural network, unless you evolve it through an algorithm like NEAT (which I chose not to get into), is decided by the user. It could have as few as a layer and as few neurons as the outputs the network needs to produce (four in this case, red, green, blue and alpha for each pixel). Potentially it can have hundreds of layers and millions of neurons. At the core of the neural network are neurons: the nodes that receive values either coming from outside of the neural network or from the previous layer. The nodes then apply a magical mathematic thing called an activation function, that transmogrifies the input value to some other. The following image represents some activation functions.
Each node has a single activation function (there might be variations on this I’m not aware of), and each of them produces different visible results on the decision. Whether or not they help will depend on how their decision is rated, and whether they get selected out through evolution.
That’s as much of an explanation as I care to give on the subject. In any case, implementing a neuroevolutionary algorithm almost from scratch on Rust took some shit: I had to get deep into generics and closures, which I had feared dealing with through my previous project. To destroy my fears and out of general obsessive-compulsiveness, I followed the Jurassic Park principle: I was so focused in whether or not I could that I didn’t stop to think if I should. Generics help you use potentially infinite variations of each part involved in a process. In this case I didn’t use any variation: you use a single type of population, genome, neural network, layer and node, and yet I made it so I can expand the system at any point of the future. Any cloner of the repository can do so as well (or should be able to anyway). The cost of that is that the client has to write code like this in main:
Which in turn looks something like this in the corresponding structures (this is all the code for training any population, without the intentions of the client code):
The following are the declarations of the types and such for the controller that deals with training a population:
Pictured: the shocking need to learn algebra.
As I was writing all the generic functions, I wondered where the border between concretion and abstraction was. Generics have no runtime performance penalties in Rust: the compiler turns all generics into concrete implementations during compilation. I struggled to understand how to deal with the inevitable points where you just had to create a concrete class (neurons, populations, neural networks), and there’s no way to create something concrete from generics. The solution ended up being obvious, but it required a different mindset: you need to inject the production of concrete structures into the generic frame. Still, how on Earth would you inject something that produces concretion, when the specific details of whatever you need to construct isn’t known most of the time until the structures need to be created?
The solution is closures. You just pass functions as parameters to the generic core. It was obvious, but it’s such a high level, duck-typing-mindset solution used in languages like Python or Javascript that they seemed impossible to implement in such a hardcore language like Rust. But it’s tremendously easy: the type of a function/closure parameter is something like Fn(u32) -> u32, meaning it receives an unsigned integer and spits out an unsigned integer, and as the value of that parameter you could pass something like |value| { value } (or even without the brackets). That’s it. Value gets passed as a parameter whenever the generic code uses the function, and the client doesn’t need to worry about anything else.
That’s about as much as I can think to write right now. I’ll soon release the programs to evolve images, and I’ll likely play “Crusader Kings 3” for a few days.
No comments have been added yet.
