Scoring Hypotheses from Threat Detection Technologies: Analogies to Machine Learning Evaluation

Contents

We have developed efficient methods to score structured hypotheses from technologies that fuse evidence from massive data streams to detect threat phenomena. We have generalized metrics (precision, recall, F-value, and area under the precision-recall curve) traditionally used in the information retrieval and machine learning communities to realize object-oriented versions that accommodate inexact matching over structured hypotheses with weighted attributes. We also exploit the object-oriented precision and recall metrics in additional metrics that account for the costs of false-positive and false-negative threat reporting. We have reported on our scoring methods more fully previously; the present brief presentation is offered to help make this work accessible to the machine learning community.