We research how domain experts can develop knowledge-based systems that incorporate their expertise. Our approach is to develop a learning agent that an expert can teach by explaining it how to solve specific problems, and by critiquing agent’s attempts to solve new problems. In this paper we present an integrated set of methods that support the domain expert in refining the rules from the agent’s large knowledge base. We developed methods to focus the expert on those steps of agent’s reasoning process that require expert’s analysis. To alleviate the expert’s tendency to omit implicit details in human communication we developed two complementary rule analysis methods that guide the expert to provide more explanations of the rule’s examples. We developed methods that analyze the expert’s critique of an example and selects the best rule refinement strategy. We developed a lazy rule refinement method that allows the modification of a learned rule, or the learning of a closely related rule, without requiring the expert to perform an analysis of the rule’s representative examples at the time of the modification. Instead, this analysis is postponed until the agent applies the rule in problem solving.