Scoring Hypotheses from Threat Detection Technologies: Analogies to Machine Learning Evaluation

Robert C. Schrag, Masami Takikawa

We have developed efficient methods to score structured hypotheses from technologies that fuse evidence from massive data streams to detect threat phenomena. We have generalized metrics (precision, recall, F-value, and area under the precision-recall curve) traditionally used in the information retrieval and machine learning communities to realize object-oriented versions that accommodate inexact matching over structured hypotheses with weighted attributes. We also exploit the object-oriented precision and recall metrics in additional metrics that account for the costs of false-positive and false-negative threat reporting. We have reported on our scoring methods more fully previously; the present brief presentation is offered to help make this work accessible to the machine learning community.

Subjects: 12. Machine Learning and Discovery; 1. Applications

Submitted: May 15, 2007

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.