Simultaneous Team Assignment and Behavior Recognition from Spatio-Temporal Agent Traces

Proceedings of the AAAI Conference on Artificial Intelligence, 21

Volume

Technical Papers

Multiagent Systems

This paper addresses the problem of activity recognition for physically-embodied agent teams. We define team activity recognition as the process of identifying team behaviors from traces of agent positions over time; for many physical domains, military or athletic, coordinated team behaviors create distinctive spatio-temporal patterns that can be used to identify low-level action sequences. This paper focuses on the novel problem of recovering agent-to-team assignments for complex team tasks where team composition, the mapping of agents into teams, changes over time. Without a priori knowledge of current team assignments, the behavior recognition problem is challenging since behaviors are characterized by the aggregate motion of the entire team and cannot generally be determined by observing the movements of a single agent in isolation. To handle this problem, we introduce a new algorithm, Simultaneous Team Assignment and Behavior Recognition(STABR), that generates behavior annotations from spatio-temporal agent traces. STABR completely annotates agent traces with 1) the correct sequence of low-level actions performed by each agent and 2) an assignment of agents to teams over time. Our algorithm employs a randomized search strategy, RANSAC, to efficiently identify candidate team assignments at selected timesteps; these hypotheses are evaluated using dynamic programming to derive a parsimonious explanation for the entire observed spatio-temporal sequence. The proposed approach is able to perform accurate team behavior recognition without an exhaustive search over the combinatorial space of potential team assignments. Experiments on simulated military maneuvers demonstrate that STABR outperforms spatial clustering, both in assignment and recognition accuracy.