In many optimization and decision problems the objective function can be expressed as a linear combination of competing criteria, the weights of which specify the relative importance of the criteria for the user. We consider the problem of learning such a "subjective" function from preference judgments collected from traces of user interactions. We propose a new algorithm for that task based on the theory of Support Vector Machines. One advantage of the algorithm is that prior knowledge about the domain can easily be included to constrain the solution. We demonstrate the algorithm in a route recommendation system that adapts to the driver’s route preferences. We present experimental results on real users that show that the algorithm performs well in practice.