Evaluating Human-Consistent Behavior in a Real-time First-person Entertainment-based Artificial Environment

May 2003

Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2003)

Volume

Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2003)

All Papers

A two-part method for objectively evaluating human and artificial players in the Quake II entertainment-based environment is presented. Evaluation is based on a collected set of twenty human player trials over a developed reference set of one hundred unique and increasingly difficult levels. The method consists of a calculated Time-Score performance measure and a k-means based clustering of player performance based on edit distances from derived player graphs. Understanding human and agent performance through this set of performance and clustering metrics, we tested this evaluation method utilizing our CAMS-DCA (Cognitivebased Agent Management System-D'Artagnan Cognitive Architecture) agents for human performance and consistency.