Teaching a Neural Net: Bipolar XOR -

i'm trying to teach neural net of 2 inputs, 4 hidden nodes (all in same layer) , 1 output node. binary representation works fine, have problems bipolar. can't figure out why, total error converge same number around 2.xx. sigmoid 2/(1+ exp(-x)) - 1. perhaps i'm sigmoiding in wrong place. example calculate output error should comparing sigmoided output expected value or sigmoided expected value?

i following website here: http://galaxy.agh.edu.pl/~vlsi/ai/backp_t_en/backprop.html , use different functions instructed use. when did try implement functions still ran same problem. either way stuck half time @ same number (a different number different implementations). please tell me if have made mistake in code somewhere or if normal (i don't see how be). momentum set 0. common 0 momentum problem? error functions supposed using are:

if ui output unit

error(i) = (ci - ui ) * f'(si )

if ui hidden unit

error(i) = error(output) * weight(i output) * f'(si)

public double sigmoid( double x ) {     double fbipolar, fbinary, temp;     temp = (1 + math.exp(-x));     fbipolar = (2 / temp) - 1;     fbinary = 1 / temp;     if(bipolar){         return fbipolar;     }else{         return fbinary;     }  }  // initialize weights random values. private void initializeweights(double neg, double pos) {      for(int = 0; < numinputs + 1; i++){         for(int j = 0; j < numhiddenneurons; j++){             inputweights[i][j] = math.random() - pos;             if(inputweights[i][j] < neg || inputweights[i][j] > pos){                 print("error ");                 print(inputweights[i][j]);             }         }     }     for(int = 0; < numhiddenneurons + 1; i++){         hiddenweights[i] = math.random() - pos;         if(hiddenweights[i] < neg || hiddenweights[i] > pos){             print("error ");             print(hiddenweights[i]);         }     } }  // computes output of nn without training. i.e. forward pass public double outputfor ( double[] arginputvector ) {      for(int = 0; < numinputs; i++){         inputs[i] = arginputvector[i];     }     double weightedsum = 0;     for(int = 0; < numhiddenneurons; i++){         weightedsum = 0;         for(int j = 0; j < numinputs + 1; j++){             weightedsum += inputweights[j][i] * inputs[j];         }         hiddenactivation[i] = sigmoid(weightedsum);      }      weightedsum = 0;     for(int j = 0; j < numhiddenneurons + 1; j++){         weightedsum += (hiddenactivation[j] * hiddenweights[j]);     }      return sigmoid(weightedsum); }      //computes derivative of f public static double fprime(double u){     double fbipolar, fbinary;     fbipolar = 0.5 * (1 - math.pow(u,2));     fbinary = u * (1 - u);     if(bipolar){         return fbipolar;     }else{         return fbinary;     } }  // method used update weights of neural net. public double train ( double [] arginputvector, double argtargetoutput ){     double output = outputfor(arginputvector);     double lastdelta;      double outputerror = (argtargetoutput - output) * fprime(output);      if(outputerror != 0){         for(int = 0; < numhiddenneurons + 1; i++){             hiddenerror[i] = hiddenweights[i] * outputerror * fprime(hiddenactivation[i]);             deltahiddenweights[i] = learningrate * outputerror * hiddenactivation[i] + (momentum * lastdelta);             hiddenweights[i] += deltahiddenweights[i];         }          for(int in = 0; in < numinputs + 1; in++){             for(int hid = 0; hid < numhiddenneurons; hid++){                 lastdelta = deltainputweights[in][hid];                 deltainputweights[in][hid] = learningrate * hiddenerror[hid] * inputs[in] + (momentum * lastdelta);                  inputweights[in][hid] += deltainputweights[in][hid];             }         }     }      return 0.5 * (argtargetoutput - output) * (argtargetoutput - output); } 

general coding comments:

initializeweights(-1.0, 1.0); 

may not initial values expecting.

initializeweights should have:

inputweights[i][j] = math.random() * (pos - neg) + neg; // ... hiddenweights[i] = (math.random() * (pos - neg)) + neg;  

instead of:

math.random() - pos; 

so works:

initializeweights(0.0, 1.0); 

and gives initial values between 0.0 , 1.0 rather between -1.0 , 0.0.

lastdelta used before declared:

deltahiddenweights[i] = learningrate * outputerror * hiddenactivation[i] + (momentum * lastdelta); 

i'm not sure if + 1 on numinputs + 1 , numhiddenneurons + 1 necessary.

remember watch out rounding of ints: 5/2 = 2, not 2.5! use 5.0/2.0 instead. in general, add .0 in code when output should double.

most importantly, have trained neuralnet long enough?

try running numinputs = 2, numhiddenneurons = 4, learningrate = 0.9, , train 1,000 or 10,000 times.

using numhiddenneurons = 2 "stuck" when trying solve xor problem.

see xor problem - simulation