The successful application of data mining techniques ideally requires both system support for the entire knowledge discovery process and the right analysis algorithms for the particular task at hand. While there are a number of successful data mining systems that support the entire mining process, they usually are limited to a fixed selection of analysis algorithms. In this paper, we argue in favor of extensibility as a key feature of data mining systems, and discuss the requirements that this entails for system architecture. We identify in which points existing data mining systems fail to meet these requirements, and then describe a new integration architecture for data mining systems that addresses these problems based on the concept of "plug-ins." KEPLER, our data mining system built according to this architecture, is presented and discussed.