Manifold Representations for Value-Function Approximations

Contents

We present a novel approach to modeling the reinforcement learning value function using a manifold representation. By explicitly modeling the topology of the value function domain, traditional problems with discontinuities and resolution can be addressed without resorting to complex function approximators. We describe the mathematical underpinnings of our approach, and show how manifold techniques can be applied to value-function approximation. We also present techniques for constructing a manifold representation of the domain, and show their effectiveness on example problems.