So, I left off last time with a brief explanation of what I want to do (clustering-wise), and then continued with why that won't work like I want.

So now, some more concepts.

I'm now limited by my limited knowledge of systems administration to a lam/mpi cluster. Not a shabby thing; MPI is a very powerful tool, and is much easier to configure than a mosix cluster.

The plan then? Worker apps!

Worker Applications

So, you've got this root node and these worker nodes. All process are created at runtime and stay in existence throughout the run of whatever it is you're doing. With GA, to mimic the work I did in mosix clusters, I would usually want to create a thread to compute each chromosome. Unfortunately, this isn't so hot for beowulf, cause that means creating popSize*numGens threads on startup. So instead, I creat a worker application, and run an instance of it on each worker node. This node will sit in an endless loop, like a server application, waiting for chromosomes to be passed to it, and it will then pass back the cost or value or fitness, whatever you like to call it, of the chromosome it evaluated, signalling that it can now accept a new chromosome.

The Problem

With mosix-style clustering, forking a process automatically creates a new process with all the chromosome information it needs, and then it needs to pass back a single variable (usually a float). With MPI style clustering, you have to pass all of the genetic material to the worker application using mpi_send() commands, much like with sockets, which means you have to specify the size of the data being transmitted, which in the case of large dynamic arrays for chromosomes means variable sized data transmissions are being sent and received, which makes your sending (and worker application) problem specific. This means that you're not writing library code (which means that it can be used for any old type of genetic algorithm problem), but instead writing problem specific code that you would have to one-off to make work for another problem.

So, I don't really like this plan much at all. I've also noticed that I've started talking alot about the genetic algorithm, so my next post will be a breif runthrough of that.

Advertisements