This paper addresses the exception-based refinement of the knowledge base of a learning agent. The knowledge base consists of an ontology that represents the objects from an application domain, and a set of problem solving rules learned from examples of problem solving episodes. Because the ontology is incomplete and is used as the generalization hierarchy for learning, the learned rules generally contain exceptions. The paper proposes a suite of interactive methods for extending the object ontology with new objects and features in order to eliminate the exceptions of the rules. Some are simpler methods for a subject matter expert, while others are more complex ones, requiring the participation of a knowledge engineer. Each method has four major phases: discovery of promising ontology refinement candidates; selection of the candidate; elicitation and refinement of the ontology based on the selected candidate; and refinement of the rules based on the extended ontology. These methods are integrated into the Disciple learning agent shell.