The starting point of this position paper is the observation that robot learning of tasks, when done autonomously, can be conveniently divided into three learning problems. In the first we must derive a controller for a task given a process model. In the second we must derive such a process model, perhaps in the face of hidden state. In the third we search for perceptual processing functions that find natural regularities in the robot’s sensory sequence, and which have utility in so far as they ease the construction of process models. We sketch an algorithm which takes a stochastic process approach to modelling, and combines methods for solving each of these three problems.