Is this Model Possible in Simbrain ?

[Neural_Wannabe]

First, i gotta give a huge shout out to the simbrain devs for making such a beautiful piece of software,
but to then give it away free so us neural noobs can play with it is way too generous. Thank You

I know there are many other programs similar to this i have tried, but this one just feels like it does things the right way.
Its mind boggling what you could actually achieve with something as powerful as this, if only i could try what i want to do lol.

I have been trying to find a way to duplicate this model in simbrain now for 2 weeks, hence my question. Is it possible to do in simbrain ?

Source : http://www.crest.fr/ckfinder/userfiles/files/pageperso/vcottet/paper%20-%20Deep%20Learning%20predicts%20Loto.pdf

I have watched all youtube vids & read the documentation, but being a total novice is this way over my head ?
Basically i love playing keno (20 numbers drawn from 80 total, only need 10 predicted) on the weekends with my mates and would love to find a bunch of sets of numbers to play with that can been predicted by training.

I have up to 15,000 previous draws to train with and believe as the paper states that : a powerful model from the neural
network framework will capture a non-random structure, even tho everyone says its not possible.

Here is the small 200 draw sample ive been working with trying to make the model:
http://s000.tinyupload.com/index.php?file_id=81226481821239112301

If anyone can help with the model or give me some direction as to how i input the formulas and add Adam (an algorithm for first-order gradient-based optimization) would be eternally grateful, as i could then have a basis for implementing some other ideas to add for fine tuning. Thanks

[jyoshimi]

Hello, and thanks for the kind words.

Deep learning like in the paper you refer to is one thing we're working on for Simbrain 4.  We've just "broken ground" on this, and it's exciting, but hard to predict how long it will be before it's released.

There may be a way to get this to work in a regular backprop network.   Can you say more about the inputs and outputs?  What do you imagine the inputs to the network   would be, and what would the outputs be?  Something like a history of 20-number draws (so some multiple of 20 in the input layer) and then a predicted next draw (so 20 numbers in the output)?

If there is indeed some non-random element, then a neural network should in principle be able to pick it up.  Simbrain 3.02 might  be up to the job, but I'm not sure...

Cheers,

- Jeff

[Neural_Wannabe]

Hey Jeff, thank you for the reply and great to hear you guys are making progress on a variety of
applications Simbrain could be used for in future released versions.

A simple Backprop Network was the first model i tried with my data when i discovered Simbrain, but
being a total noob to neural nets found that the data i had trained didn't really predict much
after checking it with the test data, and pretty much gave up on Backprop after reading & watching
a bunch vids that it was now old school & not used anymore for the type of predictive modelling i
was trying to achieve.

Basically i am trying to do a model & optimizer recreated in Simbrain to train it with my data, so
when i input a Keno game (20 numbers) it will try & predict 10 to 15 out of 80 numbers available,
the trained model thinks will be drawn next.... like in the paper.

For eg: I have a 15000 keno draw data set sample for testing a model:

http://s000.tinyupload.com/index.php?file_id=07337797027300722876

70% of data samples i use for training and 30% for testing

20 input & 15 output , In Training, error is minimized if all 15 predicted samples are close to any
of the 15 samples in a data row of csv file.

A picture tells a thousand words, here is a mockup of what it should sorta look like :



I know exactly what i want to do & this is a great way for me to learn a bit about Neural Nets,
But yeah stumped on how to make the model used in the paper & how to add the Adam optimizer or
equivalent and add those equations to the model. I have a couple of guys helping me out atm, but no
luck yet, looking like i need someone with a PhD to get anywhere with this lol.



Trying to work out the cross-presence matrix and what the convolutional layers should look like and
how to add residual learning as an intermediate result to the output of the convolutional layers,
not to mention the 2 Dense layers and non-linear sigmoid used to predict, is way over my head, but
im willing to learn, but really only want to use Simbrain as the other apps are too difficult or
expensive for me to use as this is only an entertainment project for me.

I realize that a CNN works on images usually, but maybe this is the part of how the model finds the
patterns as described in the paper. IDK, maybe im wrong.

Anyway, any help would be greatly appreciated and will post here if the 2 guys i have come thru
with some news on this.

Some Trivia about Keno you might like

Keno is a electronic lottery game where there are total of 80 numbers (1 to 80) and 20 of those
numbers a drawn per game randomly.

The probability of the model predicting 10 out of the 20 numbers drawn from the (80) is 8,911,711 to 1

The probability of the model predicting 15 out of the 20 numbers drawn from the (80) is 41,751,454 to 1 ,
in human terms if i played the prediction at 1 game per minute, 24 hours a day , 365 days a year ,
it would take 79 years to get 15 out 20.

The probability of 20-out-of-20 is 3.5 Quintillion to 1, lol

Thanks for taking the time to read this long winded reply. Cheers Joe

[Neural_Wannabe]

Edit: If Deep Learning is not an option with Simbrain in its current form, sorry for asking as it just looked like it has all the elements i needed to implement the model in that paper and that i didn't know how to use them, no worries as maybe there are other ways to model a network for what im trying to do while waiting for a possible Simbrain 4 in the future. Cheers Joe

[jyoshimi]

Hi Joe, I want to post something quickly and then may try to post more later (I'm always busy, but unusually busy right now).  First, nice pic and thanks for the background information.  Second, I had another look at that paper and saw that it was meant as a bit of humor (note the publication date).  Still, your question is worth responding to.    Basically the kind of thing you want to do is not possible in Simbrain  3.02.  I mean, it might be logically possible with scripting, but not with the kind of accessible, GUI oriented, component-wise   approach we've tried to achieve in Simbrain.  Achieving that kind of interface with modern machine learning components will take a fair bit of work.   In the mean time, I think tensor-flow might the closest thing out there to a visually oriented component-wise approach to deep learning, but if there are others I'd love to see them!  You still have to write python code, but python's a pretty nice language, and then you get a nice visualization in tensorboard.

https://www.tensorflow.org/get_started/get_started

https://www.youtube.com/watch?v=2FmcHiLCwTU

Cheers

- Jeff

[Neural_Wannabe]

Hey Jeff, appreciate your detailed reply & suggestions. I know you are busy & no worries if you
cant post, no problem.

Hahaha yes i realize the paper is meant to be in humor, but i think they were on the right track
anyway with the model & optimizer to find non-random patterns from seemingly rnd data,
ofcourse the real world data they were using wouldn't work like you would know.

Can't tell you how many times ive tried to get Tensorflow working for me lol, all dependencies
install fine, so does python & its dependencies and then tensorflow just wont install properly
throwing up a bunch of errors everytime i try the CPU or GPU version including trying to install
using Anaconda. This guy gets pretty close to showing me exactly what im trying to achieve,
except he is using images, whereas im trying to find patterns in data which could be converted
into images, but then i'm doing the work im trying the NN to find for me lol:

https://www.youtube.com/watch?v=u4alGiomYP4 

Anyway enough of my dramas with that app & dont really think i could manage
to script in Python what i needed anyway, but thanks for the suggestion.

I have been doing something similar to what i want to do using macros in excel for the longest
time, but it is that slow that i'll be dead & gone before it would ever find anything significant
hehehe.

Anyhow, i will persist with Simbrain as it is, as im enjoying using it and muck around with all
of its options see i get lucky & find something i can use for the task i need. Will Let ya know
if im successful or the other 2 guys come thru with something.

Again Appreciate your reply. Cheers Joe