optimum training results

[Ray]

Hi there,
I'm quite a newbie with this neural networks, and Simbrain finally opened up the topic, and I get an understanding of it. Pretty cool software, and very intuitive.
I'm just struggling with one thing:
When training a network with a lot of input neurons and thus a lot of connections, training can take quite some time. So I thought to run overnight. The next morning, the results (i.e. the error) were much worse than the evening before (quite stable though).
So I thought whether there is a way to somehow save the values for the connections wheen the error has reached an absolute minimum?

Thanks for some feedback, and, again, great piece of software!

Ray

[jyoshimi]

Hi there!   Yes that does happen sometimes that the error will start to go up.    The issue is related to our bare-bones implementation of backprop.    An implementation that used all the latest and greatest advances in backprop would probably not have error going back up so much, and would also be much faster.   I suggest for backprop, at least for now, using Simbrain to get a feel for how it works, but then using some other software if you have any kind of largeish task.   You could try Weka for example, or if you are comfortable with python etc. Keras or Lasagne.    If others can report on what's worked for them that would help.

I don't think improved backprop will be in Simbrain until next summer, if then.  We have quite a backlog of new things we're working on, including 3d and some image processing, and also some refactoring behind the scenes, so unless someone decides to take it on (which would be great!) it will be at least a year.  Until then, thoughts on how an improved backprop should look in Simbrain would be welcome!

Oh, as to your question of finding the minimum and saving it, perhaps there is a way to do that (the naive solution of just recording the weights at every time step could get pretty expensive).  But the real solution would be to improve the backprop stuff.

- Jeff

[Ray]

Thanks Jeff.
I had a look at Weka - just proofed how intuitive Simbrain is!
I haven't used phython yet, but maybe have a look at it as well and then try the other tools you mentioned.
If anybody has any other suggestions, let me know!

Thanks,
Ray