Overfitting

• The standard problem is overfitting.
• That is, the system in essence memorizes the training set.
• It then does not generalize to the test set.
• This problem can be reduced by increasing the error threshold,
• or by reducing the number of neurons.
• It also trains faster with fewer neurons.
• It's not really known how to select the number of neurons, transfer functions, or number of layers.
• People train with a train and a validation data set. This can reduce overfitting.
• Also, it's often not clear what the weights mean.