machine learning - Neural network weighting -

recently i've studied backpropagation network , have done manual exercise. after that, came question( maybe doesn't make sense): there thing important in following 2 different replacement methods: 1. incremental training: weights updated once delta wij's known , before presenting next training vector. 2. batch training: delta wij's computed , stored each exemplar training vector. however, delta wij's not used update weights. weight updating done @ end of training epoch.

i've googled while haven't found results.

so referring 2 modes perform gradient descent learning. in batch mode, changes weight matrix accumulated on entire presentation of training data set (one 'epoch'); online training updates weight after presentation of each vector comprising training set.

i believe consensus online training superior because converges faster (most studies report no apparent differences in accuracy). (see e.g., randall wilson & tony martinez, the general inefficiency of batch training gradient descent learning, in neural networks (2003).

the reason why online training converges faster can follow curves in error surface on each epoch. practical significance of can use larger learning rate (and therefore converge fewer cycles through training data).

put way, accumulated weight change batch training increases size of training set. result batch training uses large steps @ each iteration, , therefore misses local minima in error space topology--your solver oscillates rather converges.

batch training 'default' (most used in ml textbooks, etc.) , there's nothing wrong using long converges within acceptable time limits. again, difference in performance (resolution, or classification accuracy) small or negligible.