AAAI Publications, Twenty-Second International FLAIRS Conference

Font Size: 
Training to a Neural Net's Inherent Bias
Steven Gutstein, Olac Fuentes, Eric Freudenthal

Last modified: 2009-03-16


A neural net with multiple output nodes is capable of distinguishing among a set of related input classes even in the absence of training. It can do so with an accuracy that is markedly better than random guessing. This is because each class will tend to activate a different set of output nodes. We refer to this tendency as the net's 'inherent' bias. Ascertaining a net's inherent biasĀ  may be thought of as learning the net. One may learn the net either instead of training it, or prior to training it. Furthermore, one only needs a small number of samples from each input class in order to reliably learn the net. If a net has been previously trained on a different, related set of classes, then ascertaining the inherent bias is a form of knowledge transfer. When such a net is trained to respond in accordance with its inherent bias, one may obtain substantially higher accuracies than is provided by nets trained in the standard fashion. Furthermore, when using a deep net, we were able to obtain such improvements while only allowing the top layer of the net to train. This layer contained only about 5.7% of the net's free parameters.

Full Text: PDF