Getting Order Independence in Incremental Learning

Training Issues in Incremental Learning

Volume

Training Issues in Incremental Learning

Contents

It is empirically known that most incremental learning systems are order dependent, i.e. provide results that depend on the particular order of the data presentation. This paper aims at uncovering the reasons behind this, and at specifying the conditions that would guarantee order independence. It is shown that both an optimality and a storage criteria are sufficient for ensuring order independence. Given that these correspond to very strong requirements however, it is interesting to study necessary, hopefully less stringent, conditions. The results obtained prove that these necessary conditions are equally difficult to meet in practice. Besides its main outcome, this paper provides an interesting method to transform an history dependent bias into an history independent one.