Fast Feature Selection for Linear Value Function Approximation

  • Bahram Behzadian University of New Hampshire
  • Soheil Gharatappeh University of New Hampshire
  • Marek Petrik University of New Hampshire


Linear value function approximation is a standard approach to solving reinforcement learning problems with large state spaces. Since designing good approximation features is difficult, automatic feature selection is an important research topic. We propose a new method for feature selection that is based on a low-rank factorization of the transition matrix. Our approach derives features directly from high-dimensional raw inputs, such as image data. The method is easy to implement using SVD, and our experiments show that it is faster and more stable than alternative methods.