Exploring Massive Learning via a Prediction System

Contents

We describe the functionality of a large scale system that, given a stream of characters from a rich source, such as the pages on the web, engages in repeated prediction and learning. Its activity includes adding, removing, and updating connection weights and category nodes. Over time, the system learns to predict better and acquires new useful categories. In this work, categories are strings of characters. The system scales well and the learning is massive: in the course of 100s of millions of learning episodes, a few hours on a single machine, hundreds of thousands of categories and millions of prediction connections among them are learned.