Effects of Query and Database Sizes on Classification of News Stories Using Memory-Based Reasoning

Case-Based Reasoning and Information Retrieval: Exploring Opportunities for Technology Sharing

Volume

Papers from the 1993 AAAI Spring Symposium

Contents

In this paper we explore the effects of query and database size on news story classification performance. Memory Based Reasoning (MBR) (a k-nearest neighbor method) used as the classification method. There are 360 different possible codes. Close matches to a new story are found using an already coded training database of about 87,000 stories from the Dow Jones Press Release News Wire, and a Connection- Machine Document Retrieval system (CMDRS, [Stanfill]) that supports full text queries, as the underlying match engine. By combining the codes from the near matches, new stories are coded with a recall of about 80% and precision of about 70%, as reported in [Masand]. When the query size is varied from 10 terms to more than 200 terms (matched against the full-text of documents) the recallprecision product changes from 0.53 to 0.6. While this is a significant change, we find that moderate sized queries of 40-80 terms can suffice for finding relevant matches for classification. By changing the size of database from 10,000 stories to 80,000 we found that the recall-precision product changed from 0.22 to 0.57. This shows that with our current MBR approach the database size can’t be reduced significantly without compromising performance. We also find that fewer number of retrieved matches are needed with larger query and database sizes.