Incrementally learning new classes with generative classification

Learning continually from non-stationary streams of data is a key feature of natural intelligence, but an unsolved problem in deep learning. Particularly challenging for deep neural networks is the problem of ""class-incremental learning"", whereby a network must learn to distinguish classes that are not observed together. In deep learning, the default approach to classification is learning discriminative classifiers. This works great in the i.i.d. setting when all classes are available simultaneously, but when new classes must be learned incrementally, successful training of discriminative classifiers depends on workarounds such as storing data or generative replay. In a radical shift of gears, here I propose to instead address class-incremental learning with generative classification. Key advantage is that generative classifiers – unlike discriminative classifiers – do not compare classes during training, but only during inference (i.e., when making a classification decision). As a proof-of-concept, in preliminary work I showed that a naïve implementation of a generative classifier, with a separate variational autoencoder model per class and likelihood estimation through importance sampling, outperforms comparable generative replay methods. To improve the efficiency, scalability, and performance of this generative classifier, I propose four further modifications: (1) move the generative modelling objective from the raw inputs to an intermediate network layer