Primer for neural networks

[Brajesh]

Hi,

I'm completely new to neural networks.
I went over the simbrain tutorials and I liked them on how to use simbrain.  
It seems like a fantastic and very flexible piece of software!

Now I'd like to know more about neural networks before I start using simbrain.
I'm specifically  interested in getting a neural network to understand words and sentences.

What is a good primer to learn about neural networks, especially in my area of interest?

thank you,

Brajesh

[jyoshimi]

Hi Brajesh,

One of the first things to deal with when analyzing linguistic data with a neural network is how to convert linguistic items likes words and sentences to lists of numbers (vectors) so that networks can process them ("word embedding").  For an overview of this topic you could try googling "word2vec".

In the earlier connectionist literature there are Elman's classic papers (e.g. Finding Structure in Time), Seidenberg's triangle model, and some of the chapters in the PDP volumes, including Smolensky's, which evolved in to optimality theory, which is now dominant in phonology according to my colleagues.  Here is a review I just found.

Simbrain 3 has kind of primitive word embedding capabilities (no word2vec yet).  I'm actually planning to finally make a new youtube video, and i will cover this.  Hopefully in the next month.

Also more broadly on the topic of tutorials, I am teaching a course using Simbrain, and have co-written an open-content book on the topic. I'm still polishing it up but I hope to release it as free modifiable text on a creative commons license maybe around December.  

Cheers,

- Jeff

[Brajesh]

Super Jeff thank you,

I have some material now to get me started.
That paper looks very interesting (and a little intimidating).
I think it will give me the basics that I'm looking for.

I can't wait to see the new youtube video for word embedding capabilities!
And if it's possible to get a draft copy of your upcoming book I'd enjoy perusing it too.

I'll come back with more questions as I gain more knowledge, thank you for the reply :)

[Brajesh]

Hello Jeff,

How do I access the reference papers for the paper you provided the link to?
I found the Elman and Seidenberg via google.

I'm not an academian, but I would love to read many of the papers that were referenced.

Also, are there any books that you could also recommend for learning more about connectionism?

thank you so much, I'm off to a start.

[jyoshimi]

For papers you are kind of stuck with what's publicly available.  But there is a lot publicly available so that should keep you going for a while.  Also you can often email the authors and ask for a copy.

For introductions I like my own, which I mentioned above :) But that's still a ways from being done so in the meantime you can search for Fausett, Haykin and Rojas (search for those names + neural network).  There are others as well.     Also there is the Goodfellow et. al. deep learning book which is quite current

http://www.deeplearningbook.org/

More advanced discussions with cool embedded javascript demos are here

https://distill.pub/

- Jeff

[Brajesh]

Thanks so much Jeff !!
And I can't wait to read yours too  when its released :)

[jyoshimi]

Oh and I forget to mention the emergent-based wiki.   I probably should have mentioned it first.

https://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main

[Brajesh]

cool :)

[jyoshimi]

I can't resist adding more.   I should publish a list somewhere.... At any rate, this one requires matlab, but is by an original PDP writer, James McClelland:

https://web.stanford.edu/group/pdplab/pdphandbook/handbook.pdf

[Brajesh]

yay - keep 'em coming !