Classifier Calibration for Multi-Domain Sentiment Classification

Proceedings of the International AAAI Conference on Web and Social Media, 4

Volume

Vol. 4 No. 1 (2010): Fourth International AAAI Conference on Weblogs and Social Media

Poster Papers

Textual sentiment classifiers classify texts into a fixed number of affective classes, such as positive, negative or neutral sentiment, or subjective versus objective information. It has been observed that sentiment classifiers suffer from a lack of generalization capability: a classifier trained on a certain domain generally performs worse on data from another domain. This phenomenon has been attributed to domain-specific affective vocabulary. In this paper, we propose a voting-based thresholding approach, which calibrates a number of existing single-domain classifiers with respect to sentiment data from a new domain. The approach presupposes only a small amount of annotated data from the new domain. We evaluate three criteria for estimating thresholds, and discuss the ramifications of these criteria for the trade-off between classifier performance and manual annotation effort.

10.1609/icwsm.v4i1.14055