Since the description of the Semantic Web paradigm in 2001, technology has been proposed to allow its deployment and use. However, there is not yet any large and widely deployed set of semantically annotated Web resources available. As a result, it is not possible to evaluate the use of the technology in a real environment, and several assumptions about how the Semantic Web should work are emerging. In order to further investigate these assumptions and the related technology, we propose a simulation and evaluation platform. The platform provides tools to create Semantic Web simulations using different technologies for different purposes, and to evaluate their performance. In this paper we introduce the model of the platform and describe the current implementation. The implementation facilitates the integration of technology for an essential operation on the Semantic Web, namely Semantic Web service discovery. We illustrate the use of the platform in a case study by implementing a Semantic Web where the Jade multi-agent platform provides the framework to describe the agents, and a number of existing Semantic Web technologies are embedded in agent behavior.