XOR network doesn't learn using backprop

[Jan Andrews]

Hi.  We can't XOR to work in simbrain using backprop, it never learns it and performs very poorly.  Why would that be?  We're pretty sure we're doing everything correctly.  Thanks for any help,

--Jan

[jyoshimi]

Hi there,

How low are you able to get error? I can get it to about .01.   I usually have to randomize a few times before it gets that low.  However there are two issues in Simbrain:

1) Sometimes I also can't train it.   Somehow randomization seems to not completely randomize the network, but permute it. So sometimes I have to just create a brand new network. This is a bug.  If someone wants to help track it down, I'd welcome the help!

2) If you want to get error below .01 you can add more hidden units.  But there is some issue where it never gets to 0.   I think this might be related to the issue described here

http://simbrain.963258.n4.nabble.com/Backprop-sin-wave-learning-td4640451.html

We will work on that this summer

-- Jeff

[Josh de Leeuw]

The lowest I can get the error is about 0.25.

Steps that I am taking:

1) Open Simbrain 3.02 on Windows 10.
2) Select Insert > New Network.
3) In Network1 Window, select Insert > Insert Network > Backprop.
4) Change number of neurons to 1 (Layer 3), 2, 2. Leave neuron type options at default values.
5) Open Edit Backprop Network window by double clicking the Backprop title in Network1 window.
6) Load xor_in_binary as input data.
7) Load xor_train as target data.
8) Click "play" button in Train window to iterate indefinitely. Error drops to 0.2539 within a few thousand iterations. Then never drops further. I have tried this ~10 times and never gotten the error to be lower than this.

Am I missing a step?

Thanks for your help!

[jyoshimi]

Hi again,

In step 8, do periodically press randomize when it does not dip down after, say 500 iterations?

- Jeff

[Jan Andrews]

We just tried that and it went as low as .05 but then climbed back up again so we're worried about using it for an actual project.

On Tue, Apr 25, 2017 at 1:38 PM, jyoshimi [via Simbrain] <[hidden email]> wrote:

Hi again,

In step 8, do periodically press randomize when it does not dip down after, say 500 iterations?

- Jeff

---

If you reply to this email, your message will be added to the discussion below:

http://simbrain.963258.n4.nabble.com/XOR-network-doesn-t-learn-using-backprop-tp4640490p4640493.html

To unsubscribe from XOR network doesn't learn using backprop, click here.
NAML

[jyoshimi]

Hi again,

You could try reducing the learning rate a bit, or even quickly stopping iteration when the error is low enough.     I can generally get error to .01 without much effort.

But yes this is basically the state of things for the current Simbrain.  It works ok for teaching (at least in  my experience), but if you are looking to use this in a research or application setting you might consider something else for now.  

- Jeff