In this paper we will address several architectural decisions in defining a software control architecture for mobile robots. Our system is a collection of control primitives that enables the development of simulations or control algorithms for autonomous agents. Its computational capabilities are determined by an object-oriented constraint-based architecture. We discuss how high level knowledge, skills, goal-driven and reactive behavior are integrated within such an architecture. Our goal is to design a framework that enables the merging of classic and reactive implementation ideas. We will show, that each such type of control can be implemented in our system. The issues of task decomposition and granularity are given special attention, as they lie at the basis of our architecture. We discuss two learning methods supported by our system. The first is based on environment exploration, while the second copes with skill acquisition. Our robot, CYCLOPS , is a LEGO mini-robot based on the 6.270 MIT kit to be used in a package delivery application.