2

I am currently writing a program that would be able to play snake on an 25*25 grid. It works by optimizing a set of weights of 300 different solutions (each solution would be a different neural network) with the aid of an evolutionary strategy, thus by random mutation and parent selection. I have decided not to apply crossover on the parents pool due to the black box problem concerning MLPs and other neural networks. My population size is 300 and 10 parents are selected every generations (200 gen. total). Every other solution which is not a parent gets deleted to make place for 290 new mutated solutions based on the parents.

My network structure (MLP) is the following : 6 input nodes, (10,10) hidden nodes, 3 output nodes.

With my current version of the programme, I am able to reach a steady score of 60 apples (or points) eaten (around 15/200 have a score above 60). And this only after 50 generations! But now, it seems like i have bumped into a local minima that I am not able to overcome. Does anyone have any idea if further progress is still possible or that i might have hit the ceiling? The quickly elevating performance might be a sign that my exploration/eploitation balance might not be ideal, but with each solution containing 190 weights, more exploration would seem too computationally expensive and would take forever on my i7 10gen laptop :/.

Can i call this a premature convergence and is there a way to deal with this, or have i hit the limits of my idea?

Nick Stevens
  • 21
  • 5
  • I fail to understand how the MLPs are trained in your program. Are they trained using back-propagation as usualy done or they are evolutionary evolved via evolutionary programming? – Nikos M. Apr 20 '21 at 16:17
  • 1
    i extract the weights as a 190 dimensional vector (190 weights for 6 input nodes, (10,10) hidden nodes and 3 output nodes). I select the best performing networks and their weight sets accordingly. Now, with a pop. of 300 and 10 parents. each parent will be copied 29 times and inserted into the population (the parents themselves get reinserted too). But while inserting them (=> the copies), i make random changes in the weights sets and then update the weights. Redoing this over different generations. I like to refer to this as a form of random local search mixed with an evolutionary strategy. – Nick Stevens Apr 20 '21 at 16:54
  • Sorry for the undeterministic description, I have never had the correct theory behind it but i think this is what u refer to as evolutionary programming. – Nick Stevens Apr 20 '21 at 16:55
  • 1
    I think unless MLP architecture changes (eg add more hidden layers, or special layers like dropouts, ..) then most probably you have hit a ceiling – Nikos M. Apr 20 '21 at 17:13
  • What would be preferred? The biggest challenge is that I add a lot more complexity to training the MLP if more hidden layers are used. I might be able to speed up some processes with code optimization, but this will cost exponentionally more time. – Nick Stevens Apr 20 '21 at 17:20
  • Both changes proposed will get you out of the minima you are in now, but of course you will hit other minima later on. So trial and error can be the best guide here. – Nikos M. Apr 20 '21 at 17:21
  • You can even try other NNs like CNNs or RNNs – Nikos M. Apr 20 '21 at 17:22
  • Could enlarging the input set work? – Nick Stevens Apr 20 '21 at 17:25
  • Yes it is possible, in fact most changes to the setup you have now will alter the minima, that includes changes to architecture, type of NNs, input set and so on – Nikos M. Apr 20 '21 at 17:27

1 Answers1

1

You can take a look at dropout, it will get you out of the local minima you are stuck in.

Abhishek Verma
  • 672
  • 3
  • 4