The task in the computer security domain of anomaly detection is to characterize the behaviors of a computer user (the "valid', or "normal' user) so that unusual occurrences can be detected by comparison of the current input stream to the valid user’s profile. This task requires an online leaming system that can respond to concept drift and handle discrete non-metric time sequence data. We present an architecture for online learning in the anomaly detection domain and address the issues of incremental updating of system parameters and instance selection. We demonstrate a method for measuring direction and magnitude of concept drift in the classification space and present approaches to the above stated issues which make use of the drift measurement. An empirical evaluation demonstrates the relative strengths and weaknesses of these techniques in comparison to a number of baseline techniques. We show that, for some users, our drift adaptive techniques are advantageous.