Classification of Composite Actions involving Interaction with Objects

Contents

We have developed a framework to recognize composite actions by training from simple interactions with objects. We start with object tracking data for the hand object interaction. We collect training data for simple actions such as pull, push, lift, drop and put. The training data consists of object translations, scale as well as object hand connectivity information. We apply transformations to merge these simple actions into composite actions such as put-push and push-lift. These composite actions are used to build a classifier using Singular Value Decomposition (SVD). We collect test data for these composite actions and use Euclidean distance for matching. Recognition performance is significantly improved using time warping techniques such as Correlation Optimized Warping (COW).