In this paper, we investigate enhancements to an upper classifier - a decision algorithm generated by an upper classification method, which is one of the classification methods in rough set theory. Specifically, we consider two enhancements. First, we present a stepwise backward feature selection algorithm to preprocess a given set of features. This is important because rough classification methods are incapable of removing superfluous features. We prove that the stepwise backward selection algorithm finds a small subset of relevant features that are ideally sufficient and necessary to define target concepts with respect to a given threshold. This threshold value indicates an acceptable degradation in the quality of an upper classifier. Second, to make an upper classifier adaptive, we associate it with some kind of frequency information, which we call incremental information. An extended decision table is used to represent an adaptive upper classifier. It is also used for interpreting an upper classifier either deterministically or nondeterministically.